Jul 06 01:26:40.466068 ip-10-0-133-193 systemd[1]: Starting Kubernetes Kubelet... Jul 06 01:26:40.872847 ip-10-0-133-193 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:26:40.872847 ip-10-0-133-193 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 01:26:40.872847 ip-10-0-133-193 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:26:40.872847 ip-10-0-133-193 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 01:26:40.872847 ip-10-0-133-193 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 01:26:40.875552 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.875477 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 01:26:40.880801 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880778 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880803 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880807 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880810 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880815 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880818 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880821 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880824 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880827 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880829 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880831 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880837 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880840 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880842 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880845 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880848 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880851 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:40.880846 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880853 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880856 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880859 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880861 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880864 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880866 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880868 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880871 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880873 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880875 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880878 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880882 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880884 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880886 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880889 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880891 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880893 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880896 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880898 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880900 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:40.881246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880903 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880906 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880909 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880911 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880914 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880918 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880921 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880924 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880926 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880929 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880931 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880933 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880936 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880938 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880941 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880943 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880946 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880948 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880951 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880953 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:40.881722 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880956 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880959 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880961 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880965 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880968 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880970 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880973 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880975 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880978 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880981 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880985 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880988 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880991 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880994 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.880997 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881000 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881003 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881005 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881007 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:40.882221 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881010 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881012 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881014 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881018 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881021 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881023 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881026 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881029 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881031 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881034 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881037 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881039 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881042 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881044 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881047 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881049 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881051 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881054 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:40.882677 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881056 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881059 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881061 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881063 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881066 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881717 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881725 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881728 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881731 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881734 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881737 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881740 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881742 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881745 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881747 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881750 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881753 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881755 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881757 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881760 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:40.883130 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881762 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881765 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881768 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881770 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881775 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881778 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881781 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881783 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881804 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881809 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881814 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881821 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881825 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881828 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881830 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881832 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881835 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881837 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881842 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:40.883626 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881844 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881846 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881849 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881852 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881855 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881858 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881860 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881862 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881865 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881867 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881870 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881872 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881874 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881877 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881880 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881882 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881884 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881887 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881889 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881892 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:40.884099 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881894 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881897 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881901 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881904 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881909 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881912 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881914 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881917 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881919 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881922 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881924 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881928 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881930 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881933 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881935 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881938 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881940 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881949 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881952 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881954 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:40.884586 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881957 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881959 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881962 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881965 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881968 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881970 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881972 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881975 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881977 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881980 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881984 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881987 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881990 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881993 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881996 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.881998 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882001 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882003 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882005 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882007 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:40.885082 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882010 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882012 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882015 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882020 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882022 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882390 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882396 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882399 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882402 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882405 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882408 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882411 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882413 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882416 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882418 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882421 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882424 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882427 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882429 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882432 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:40.885580 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882434 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882436 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882439 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882441 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882444 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882446 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882449 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882451 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882453 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882456 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882458 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882461 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882463 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882465 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882468 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882471 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882473 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882476 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882478 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882481 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:40.886078 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882484 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882486 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882489 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882492 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882494 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882496 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882499 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882501 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882505 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882509 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882511 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882514 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882517 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882520 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882522 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882525 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882527 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882530 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882532 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:40.886554 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882537 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882540 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882543 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882545 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882547 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882550 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882552 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882555 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882557 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882560 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882563 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882565 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882567 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882569 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882572 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882574 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882577 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882580 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882583 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:40.887051 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882587 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882589 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882592 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882595 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882597 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882600 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882602 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882605 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882607 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882610 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882612 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882615 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882617 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882620 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882622 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882624 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882627 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882630 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882632 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882635 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:40.887519 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882637 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882640 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882643 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882645 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882648 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.882650 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882716 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882723 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882729 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882734 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882740 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882745 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882751 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882758 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882762 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882765 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882768 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882771 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882774 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882776 2574 flags.go:64] FLAG: --cgroup-root="" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882779 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882782 2574 flags.go:64] FLAG: --client-ca-file="" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882784 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882803 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 01:26:40.888019 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882809 2574 flags.go:64] FLAG: --cluster-domain="" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882814 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882817 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882819 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882823 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882826 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882829 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882832 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882835 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882838 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882840 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882843 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882846 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882849 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882853 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882856 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882858 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882861 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882863 2574 flags.go:64] FLAG: --enable-server="true" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882867 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882871 2574 flags.go:64] FLAG: --event-burst="100" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882874 2574 flags.go:64] FLAG: --event-qps="50" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882877 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882880 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 01:26:40.888624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882882 2574 flags.go:64] FLAG: --eviction-hard="" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882886 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882888 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882893 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882896 2574 flags.go:64] FLAG: --eviction-soft="" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882899 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882901 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882905 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882907 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882910 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882912 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882915 2574 flags.go:64] FLAG: --feature-gates="" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882919 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882921 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882924 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882927 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882932 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882935 2574 flags.go:64] FLAG: --help="false" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882938 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-133-193.ec2.internal" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882941 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882943 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882946 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882949 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 01:26:40.889287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882953 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882955 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882957 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882960 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882963 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882966 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882969 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882971 2574 flags.go:64] FLAG: --kube-reserved="" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882974 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882976 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882979 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882982 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882986 2574 flags.go:64] FLAG: --lock-file="" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882989 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882991 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882994 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.882999 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883001 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883004 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883007 2574 flags.go:64] FLAG: --logging-format="text" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883009 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883012 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883015 2574 flags.go:64] FLAG: --manifest-url="" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883018 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883021 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 01:26:40.890214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883025 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883029 2574 flags.go:64] FLAG: --max-pods="110" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883032 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883034 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883037 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883040 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883043 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883045 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883048 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883054 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883057 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883059 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883062 2574 flags.go:64] FLAG: --pod-cidr="" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883066 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883070 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883073 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883076 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883079 2574 flags.go:64] FLAG: --port="10250" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883082 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883086 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0382b5029a5f5d8f8" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883088 2574 flags.go:64] FLAG: --qos-reserved="" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883091 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883094 2574 flags.go:64] FLAG: --register-node="true" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883097 2574 flags.go:64] FLAG: --register-with-taints="" Jul 06 01:26:40.890857 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883100 2574 flags.go:64] FLAG: --registry-burst="10" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883103 2574 flags.go:64] FLAG: --registry-qps="5" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883105 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883108 2574 flags.go:64] FLAG: --reserved-memory="" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883111 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883114 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883117 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883120 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883122 2574 flags.go:64] FLAG: --runonce="false" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883127 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883130 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883133 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883135 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883138 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883141 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883144 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883147 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883149 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883152 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883154 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883157 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883160 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883163 2574 flags.go:64] FLAG: --system-cgroups="" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883166 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883170 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 01:26:40.891441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883173 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883175 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883180 2574 flags.go:64] FLAG: --tls-min-version="" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883183 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883185 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883188 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883191 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883201 2574 flags.go:64] FLAG: --v="2" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883205 2574 flags.go:64] FLAG: --version="false" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883208 2574 flags.go:64] FLAG: --vmodule="" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883211 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.883214 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883320 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883324 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883327 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883330 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883334 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883337 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883340 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883345 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883347 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:40.892054 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883350 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883354 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883357 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883360 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883363 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883367 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883369 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883372 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883375 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883379 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883382 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883385 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883388 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883392 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883395 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883397 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883400 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883403 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883406 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:40.892593 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883408 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883411 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883414 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883416 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883419 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883421 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883424 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883426 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883430 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883432 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883435 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883438 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883440 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883443 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883445 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883448 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883450 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883453 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883455 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:40.893121 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883457 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883460 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883462 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883465 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883468 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883470 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883473 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883477 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883479 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883482 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883484 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883487 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883489 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883492 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883494 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883496 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883499 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883501 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883503 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883506 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883508 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:40.893575 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883512 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883514 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883517 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883520 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883522 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883525 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883527 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883529 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883532 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883534 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883537 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883539 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883542 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883544 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883547 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883550 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883552 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883555 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883559 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883562 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:40.894098 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883564 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883566 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883569 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883572 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883574 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883577 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883579 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883581 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883584 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883586 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.883589 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.884132 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.891785 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.891816 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 01:26:40.894576 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.892656 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.892664 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892727 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892733 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892736 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892739 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892742 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892746 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892750 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892753 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892756 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892758 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892761 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892764 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892766 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892769 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892771 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892773 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892776 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:40.894940 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892778 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892781 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892784 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892786 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892805 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892810 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892813 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892816 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892820 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892823 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892827 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892830 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892833 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892836 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892839 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892841 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892844 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892846 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892849 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:40.895410 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892852 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892854 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892857 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892859 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892862 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892864 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892867 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892869 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892871 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892874 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892876 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892879 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892881 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892884 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892887 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892889 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892892 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892895 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892897 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892900 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:40.895885 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892902 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892905 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892907 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892909 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892913 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892917 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892920 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892923 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892926 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892928 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892931 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892933 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892935 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892938 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892940 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892942 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892945 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892947 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892950 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:40.896373 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892952 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892955 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892957 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892960 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892968 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892971 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892974 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892976 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892978 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892981 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892983 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892985 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892988 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892990 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892993 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892995 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.892998 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893000 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893003 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893006 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:40.896979 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893008 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893011 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893013 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893016 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.893021 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893132 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893137 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893140 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893143 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893145 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893148 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893151 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893153 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893155 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893158 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 01:26:40.897458 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893160 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893163 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893170 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893173 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893176 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893178 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893180 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893182 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893185 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893187 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893190 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893192 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893195 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893197 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893199 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893202 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893205 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893207 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893210 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893212 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 01:26:40.897826 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893215 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893217 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893220 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893222 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893225 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893227 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893230 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893233 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893235 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893237 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893240 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893242 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893244 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893246 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893250 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893259 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893261 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893264 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893267 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 01:26:40.898305 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893269 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893272 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893275 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893277 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893280 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893283 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893285 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893289 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893292 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893295 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893297 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893299 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893302 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893304 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893307 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893309 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893312 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893314 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893317 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893319 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 01:26:40.898765 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893321 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893324 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893326 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893329 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893331 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893333 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893336 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893338 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893340 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893344 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893346 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893348 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893351 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893354 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893356 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893359 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893362 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893366 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893368 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893371 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 01:26:40.899259 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893373 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893375 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893378 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893380 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893382 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893385 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893387 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893389 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893392 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:40.893394 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.893398 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.893494 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.896865 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.898164 2574 server.go:1019] "Starting client certificate rotation" Jul 06 01:26:40.899733 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.898262 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 01:26:40.900090 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.898293 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 01:26:40.900464 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.900448 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 01:26:40.901826 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.901813 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 01:26:40.915574 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.915558 2574 log.go:25] "Validated CRI v1 runtime API" Jul 06 01:26:40.918529 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.918510 2574 log.go:25] "Validated CRI v1 image API" Jul 06 01:26:40.919939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.919924 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 01:26:40.922683 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.922664 2574 fs.go:135] Filesystem UUIDs: map[65b5bfe2-58d1-4da5-8381-d7042f4551c1:/dev/nvme0n1p3 783215ee-9faa-4352-822a-5673de27ffd5:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 06 01:26:40.922752 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.922682 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 01:26:40.923777 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.923746 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 01:26:40.928190 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.928080 2574 manager.go:216] Machine: {Timestamp:2026-07-06 01:26:40.926901654 +0000 UTC m=+0.329258505 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106570 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20f7d93e6d8f709f8b3cc2bf4255ae SystemUUID:ec20f7d9-3e6d-8f70-9f8b-3cc2bf4255ae BootID:9603f4ac-0128-43d4-be7d-e9324fc735f1 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b2:c8:7e:28:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b2:c8:7e:28:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:cf:f2:cd:e2:1b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 01:26:40.928190 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.928178 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 01:26:40.928331 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.928250 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 01:26:40.931831 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.931805 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 01:26:40.931985 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.931834 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-193.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 01:26:40.932964 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.932952 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 01:26:40.933027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.932971 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 01:26:40.933027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.932980 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 01:26:40.933641 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.933630 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 01:26:40.934812 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.934783 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 01:26:40.937488 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.937475 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 06 01:26:40.937624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.937613 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 01:26:40.937680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.937652 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 06 01:26:40.937725 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.937692 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 01:26:40.937725 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.937711 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 01:26:40.937819 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.937726 2574 kubelet.go:398] "Adding apiserver pod source" Jul 06 01:26:40.937819 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.937738 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 01:26:40.938126 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.938112 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 01:26:40.938190 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.938135 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 01:26:40.940114 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.940097 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 01:26:40.942132 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.942118 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 01:26:40.942185 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.942147 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 01:26:40.943072 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943061 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943076 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943083 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943089 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943097 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943102 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943110 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943116 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943122 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 01:26:40.943131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943128 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 01:26:40.943362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943136 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 01:26:40.943362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.943143 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 01:26:40.944554 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.944538 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 01:26:40.944631 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.944557 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 01:26:40.945298 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.945282 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ddxlg" Jul 06 01:26:40.946536 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:40.946494 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 01:26:40.946536 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:40.946493 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 01:26:40.946691 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.946522 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-193.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 01:26:40.947815 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.947784 2574 server.go:1268] "Started kubelet" Jul 06 01:26:40.947924 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.947884 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 01:26:40.947985 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.947901 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 01:26:40.947985 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.947946 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 01:26:40.948636 ip-10-0-133-193 systemd[1]: Started Kubernetes Kubelet. Jul 06 01:26:40.949173 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.948955 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 01:26:40.949658 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.949646 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 06 01:26:40.952995 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.952976 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ddxlg" Jul 06 01:26:40.953707 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:40.953691 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 01:26:40.957055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.957038 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 01:26:40.957125 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.957043 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 01:26:40.957185 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:40.957129 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:40.961099 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.961083 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 01:26:40.961099 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.961087 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 01:26:40.961237 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.961110 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 01:26:40.961237 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.961190 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 06 01:26:40.961237 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.961198 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 06 01:26:40.965747 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.965730 2574 factory.go:153] Registering CRI-O factory Jul 06 01:26:40.966112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.966092 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:40.966246 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.965750 2574 factory.go:223] Registration of the crio container factory successfully Jul 06 01:26:40.966320 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.966294 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 01:26:40.966320 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.966305 2574 factory.go:55] Registering systemd factory Jul 06 01:26:40.966320 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.966315 2574 factory.go:223] Registration of the systemd container factory successfully Jul 06 01:26:40.966465 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.966337 2574 factory.go:103] Registering Raw factory Jul 06 01:26:40.966465 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.966358 2574 manager.go:1196] Started watching for new ooms in manager Jul 06 01:26:40.966959 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.966944 2574 manager.go:349] Starting recovery of all containers Jul 06 01:26:40.968876 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:40.968846 2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-193.ec2.internal\" not found" node="ip-10-0-133-193.ec2.internal" Jul 06 01:26:40.978007 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.977828 2574 manager.go:354] Recovery completed Jul 06 01:26:40.981750 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.981737 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:40.983395 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.983377 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:40.983456 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.983413 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:40.983456 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.983426 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:40.983940 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.983917 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 01:26:40.983940 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.983931 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 01:26:40.983940 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.983935 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:40.984051 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.983950 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 06 01:26:40.984573 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.984560 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:40.984640 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.984582 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:40.984640 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.984591 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:40.984904 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.984893 2574 policy_none.go:49] "None policy: Start" Jul 06 01:26:40.984944 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.984908 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 01:26:40.984944 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.984917 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 01:26:40.985850 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:40.985840 2574 policy_none.go:47] "Start" Jul 06 01:26:41.023954 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.023777 2574 manager.go:337] "Starting Device Plugin manager" Jul 06 01:26:41.023954 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.023837 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 01:26:41.023954 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.023852 2574 server.go:87] "Starting device plugin registration server" Jul 06 01:26:41.024142 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.024082 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 01:26:41.024142 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.024094 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 01:26:41.024269 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.024228 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 01:26:41.024329 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.024314 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 01:26:41.024372 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.024330 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 01:26:41.024874 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.024850 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 01:26:41.024950 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.024902 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.106842 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.106803 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 01:26:41.108140 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.108126 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 01:26:41.108234 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.108168 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 01:26:41.108234 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.108192 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 01:26:41.108331 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.108230 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 01:26:41.110910 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.110895 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:41.125040 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.124994 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:41.125940 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.125926 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:41.126022 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.125958 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:41.126022 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.125975 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:41.126022 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.126002 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.136327 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.136305 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.136412 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.136328 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-193.ec2.internal\": node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.153228 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.153206 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.208655 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.208626 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal"] Jul 06 01:26:41.208746 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.208688 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:41.209477 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.209463 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:41.209535 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.209490 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:41.209535 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.209504 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:41.210860 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.210847 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:41.211012 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.210999 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.211046 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.211027 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:41.211557 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.211542 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:41.211557 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.211553 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:41.211670 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.211571 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:41.211670 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.211575 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:41.211670 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.211585 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:41.211670 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.211590 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:41.212851 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.212835 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.212894 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.212866 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 01:26:41.213526 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.213512 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientMemory" Jul 06 01:26:41.213590 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.213532 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 01:26:41.213590 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.213546 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeHasSufficientPID" Jul 06 01:26:41.237366 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.237349 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-193.ec2.internal\" not found" node="ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.241586 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.241571 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-193.ec2.internal\" not found" node="ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.253828 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.253814 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.263305 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.263289 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ef1bea36b512041d79d06c57fe1854b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal\" (UID: \"9ef1bea36b512041d79d06c57fe1854b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.263368 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.263316 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ef1bea36b512041d79d06c57fe1854b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal\" (UID: \"9ef1bea36b512041d79d06c57fe1854b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.263368 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.263338 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ae2047bf8edfd2708b684a75d4f25bd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-193.ec2.internal\" (UID: \"4ae2047bf8edfd2708b684a75d4f25bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.354119 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.354099 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.363558 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.363539 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ef1bea36b512041d79d06c57fe1854b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal\" (UID: \"9ef1bea36b512041d79d06c57fe1854b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.363655 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.363599 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ef1bea36b512041d79d06c57fe1854b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal\" (UID: \"9ef1bea36b512041d79d06c57fe1854b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.363655 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.363562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ef1bea36b512041d79d06c57fe1854b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal\" (UID: \"9ef1bea36b512041d79d06c57fe1854b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.363655 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.363643 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ef1bea36b512041d79d06c57fe1854b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal\" (UID: \"9ef1bea36b512041d79d06c57fe1854b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.363736 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.363664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ae2047bf8edfd2708b684a75d4f25bd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-193.ec2.internal\" (UID: \"4ae2047bf8edfd2708b684a75d4f25bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.363736 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.363696 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4ae2047bf8edfd2708b684a75d4f25bd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-193.ec2.internal\" (UID: \"4ae2047bf8edfd2708b684a75d4f25bd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.455061 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.454986 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.541597 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.541570 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.544058 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.544041 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" Jul 06 01:26:41.555856 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.555837 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.656460 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.656437 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.756991 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.756946 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.831101 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.831080 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:41.857429 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.857405 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:41.897852 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.897834 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 01:26:41.898325 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.897965 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 01:26:41.898325 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.897986 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 01:26:41.898325 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.898016 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 01:26:41.957619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.957561 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 01:21:42 +0000 UTC" deadline="2028-02-09 14:38:46.275973449 +0000 UTC" Jul 06 01:26:41.957619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:41.957618 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14005h12m4.318359018s" Jul 06 01:26:41.957737 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:41.957569 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:42.058423 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:42.058370 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:42.147777 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:42.147740 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ef1bea36b512041d79d06c57fe1854b.slice/crio-fc87b1851bcf9de77fe543e9ff7d82aba075c7dd4c13e1528ff30dac3d0b9826 WatchSource:0}: Error finding container fc87b1851bcf9de77fe543e9ff7d82aba075c7dd4c13e1528ff30dac3d0b9826: Status 404 returned error can't find the container with id fc87b1851bcf9de77fe543e9ff7d82aba075c7dd4c13e1528ff30dac3d0b9826 Jul 06 01:26:42.151958 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:42.151923 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae2047bf8edfd2708b684a75d4f25bd.slice/crio-62a8f90b1e8c6dc9dd90c13be052ec11abbf8092e02d6de6200c6966d3bfcb79 WatchSource:0}: Error finding container 62a8f90b1e8c6dc9dd90c13be052ec11abbf8092e02d6de6200c6966d3bfcb79: Status 404 returned error can't find the container with id 62a8f90b1e8c6dc9dd90c13be052ec11abbf8092e02d6de6200c6966d3bfcb79 Jul 06 01:26:42.154137 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:42.154122 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:26:42.158901 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:42.158880 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-193.ec2.internal\" not found" Jul 06 01:26:42.209720 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:42.209703 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:42.257134 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:42.257109 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" Jul 06 01:26:42.267981 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:42.267966 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 01:26:42.268872 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:42.268860 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" Jul 06 01:26:42.276553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:42.276541 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 01:26:44.101002 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.100972 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:44.206167 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.206093 2574 apiserver.go:52] "Watching apiserver" Jul 06 01:26:44.211125 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.211100 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 01:26:44.211549 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.211523 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nzcfc","kube-system/konnectivity-agent-qws5x","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2","openshift-cluster-node-tuning-operator/tuned-tztdg","openshift-dns/node-resolver-6x7cb","openshift-image-registry/node-ca-9hcgl","openshift-network-diagnostics/network-check-target-l68tb","kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal","openshift-multus/multus-additional-cni-plugins-lpd78","openshift-multus/multus-qts96","openshift-multus/network-metrics-daemon-cn8h2","openshift-network-operator/iptables-alerter-hrz97","openshift-ovn-kubernetes/ovnkube-node-9pbj9"] Jul 06 01:26:44.213198 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.213175 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:44.213314 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.213258 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:26:44.214229 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.214207 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.215768 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.215734 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.216469 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.216515 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.216534 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9blcj\"" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.216705 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.217694 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.217953 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.218227 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k65tm\"" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.218522 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 01:26:44.219302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.218747 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.219769 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.219328 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.220283 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.220262 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:26:44.220398 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.220328 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.220610 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.220594 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sccwj\"" Jul 06 01:26:44.220843 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.220825 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 01:26:44.222012 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.221992 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.223392 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.223374 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 01:26:44.224154 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.224134 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 01:26:44.224387 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.224367 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 01:26:44.224455 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.224393 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 01:26:44.224670 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.224648 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mdm9w\"" Jul 06 01:26:44.224794 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.224761 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 01:26:44.224986 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.224966 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 01:26:44.225048 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.224988 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 01:26:44.225097 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.225087 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gzqbf\"" Jul 06 01:26:44.225181 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.224991 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 01:26:44.225564 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.225290 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 01:26:44.225564 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.225364 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 01:26:44.225720 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.225571 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 01:26:44.225720 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.225637 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6mg99\"" Jul 06 01:26:44.226318 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.225839 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 01:26:44.227776 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.227517 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.227776 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.227587 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:26:44.228646 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.228626 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:26:44.229925 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.229909 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.230915 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.230593 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 01:26:44.230915 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.230787 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bj66x\"" Jul 06 01:26:44.230915 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.230921 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 01:26:44.231506 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.231488 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qts96" Jul 06 01:26:44.232116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.231983 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 01:26:44.232116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.232020 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 01:26:44.232116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.232072 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x925q\"" Jul 06 01:26:44.232351 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.232334 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 01:26:44.232428 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.232367 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 01:26:44.232513 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.232480 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:44.232585 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.232549 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 01:26:44.232650 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.232546 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:26:44.233385 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.233139 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h6b4t\"" Jul 06 01:26:44.233385 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.233173 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 01:26:44.239924 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.239904 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 01:26:44.240020 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.239926 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgz6s\" (UniqueName: \"kubernetes.io/projected/d4a6fb79-4e21-48a1-b3cd-630970335d30-kube-api-access-kgz6s\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.240020 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.239956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysctl-conf\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240020 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.239978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-sys\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240020 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240003 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzvwg\" (UniqueName: \"kubernetes.io/projected/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-kube-api-access-tzvwg\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240027 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-cni-bin\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.240210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240049 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca72464b-98ac-4773-adec-7a0df6508fa7-hosts-file\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.240210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240069 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-tmp\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240107 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-cni-netd\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.240210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240135 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-os-release\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.240210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240170 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cni-binary-copy\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.240210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240184 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-host\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240210 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-tuned\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240224 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-env-overrides\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240251 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jpcd\" (UniqueName: \"kubernetes.io/projected/ca72464b-98ac-4773-adec-7a0df6508fa7-kube-api-access-4jpcd\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240275 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-node-log\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240298 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-systemd\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240346 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-systemd-units\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240364 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240389 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da1603f3-dd4b-4993-9730-75f00719033b-konnectivity-ca\") pod \"konnectivity-agent-qws5x\" (UID: \"da1603f3-dd4b-4993-9730-75f00719033b\") " pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240404 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240420 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysconfig\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240440 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j888p\" (UniqueName: \"kubernetes.io/projected/d46d6ac8-8042-4cf4-921c-313657d78764-kube-api-access-j888p\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cnibin\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240525 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-var-lib-kubelet\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240544 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-dbus\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240562 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da1603f3-dd4b-4993-9730-75f00719033b-agent-certs\") pod \"konnectivity-agent-qws5x\" (UID: \"da1603f3-dd4b-4993-9730-75f00719033b\") " pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:26:44.240583 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240588 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240606 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-kubelet\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240620 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-systemd\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-log-socket\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240664 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-system-cni-dir\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240697 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240725 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-registration-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240774 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240799 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-run\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240845 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b1a91258-14a4-45b3-ba6c-41ec9b31b731-host\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/682756aa-6265-455c-8c8e-81e95817fd8a-ovn-node-metrics-cert\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240931 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-device-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.240965 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mfgz\" (UniqueName: \"kubernetes.io/projected/9eab25ce-e60c-439c-8c4c-befdee9c1be3-kube-api-access-8mfgz\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241033 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-etc-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-ovn\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241074 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-socket-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241113 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-modprobe-d\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-kubernetes\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysctl-d\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241223 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-run-netns\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241249 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94xhv\" (UniqueName: \"kubernetes.io/projected/682756aa-6265-455c-8c8e-81e95817fd8a-kube-api-access-94xhv\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241273 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4a6fb79-4e21-48a1-b3cd-630970335d30-host-slash\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241288 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-etc-selinux\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241317 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca72464b-98ac-4773-adec-7a0df6508fa7-tmp-dir\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241335 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzq8l\" (UniqueName: \"kubernetes.io/projected/b1a91258-14a4-45b3-ba6c-41ec9b31b731-kube-api-access-tzq8l\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241385 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-slash\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241415 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-ovnkube-config\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241440 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-var-lib-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241505 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241540 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-sys-fs\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.241795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241567 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.242374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241618 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b1a91258-14a4-45b3-ba6c-41ec9b31b731-serviceca\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.242374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241642 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-ovnkube-script-lib\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.242374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241663 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4a6fb79-4e21-48a1-b3cd-630970335d30-iptables-alerter-script\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.242374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-kubelet-config\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.242374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.241720 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-lib-modules\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.257517 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.257488 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vbrwv" Jul 06 01:26:44.264880 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.264845 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vbrwv" Jul 06 01:26:44.328097 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.328074 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:44.329085 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.329026 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 01:26:44.342904 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.342873 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca72464b-98ac-4773-adec-7a0df6508fa7-hosts-file\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.343005 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.342914 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-tmp\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.343005 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.342939 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-cni-netd\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.343005 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.342961 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-os-release\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.343005 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.342984 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cni-binary-copy\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343005 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-host\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343032 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-tuned\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343055 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-env-overrides\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343083 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343107 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8r5d\" (UniqueName: \"kubernetes.io/projected/b0a348e4-0203-4974-b406-002dc6a05d58-kube-api-access-r8r5d\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343130 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-netns\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343156 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jpcd\" (UniqueName: \"kubernetes.io/projected/ca72464b-98ac-4773-adec-7a0df6508fa7-kube-api-access-4jpcd\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-node-log\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.343210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-cni-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343221 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e89584c-ddcf-48db-9695-696f9b47dea3-cni-binary-copy\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343243 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-systemd\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343265 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-systemd-units\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343289 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343313 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da1603f3-dd4b-4993-9730-75f00719033b-konnectivity-ca\") pod \"konnectivity-agent-qws5x\" (UID: \"da1603f3-dd4b-4993-9730-75f00719033b\") " pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343337 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343367 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysconfig\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343389 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-conf-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343414 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j888p\" (UniqueName: \"kubernetes.io/projected/d46d6ac8-8042-4cf4-921c-313657d78764-kube-api-access-j888p\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343437 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cnibin\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343464 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-var-lib-kubelet\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343486 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-system-cni-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343509 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-cni-bin\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343535 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-dbus\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343559 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da1603f3-dd4b-4993-9730-75f00719033b-agent-certs\") pod \"konnectivity-agent-qws5x\" (UID: \"da1603f3-dd4b-4993-9730-75f00719033b\") " pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:26:44.343599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343583 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343607 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-kubelet\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343634 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-systemd\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343659 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-log-socket\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343681 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-os-release\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343707 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-system-cni-dir\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343732 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343754 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-kubelet\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343779 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-registration-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343804 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343827 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-run\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343851 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b1a91258-14a4-45b3-ba6c-41ec9b31b731-host\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343891 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/682756aa-6265-455c-8c8e-81e95817fd8a-ovn-node-metrics-cert\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343915 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-cni-multus\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343939 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-daemon-config\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343967 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-device-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.343992 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mfgz\" (UniqueName: \"kubernetes.io/projected/9eab25ce-e60c-439c-8c4c-befdee9c1be3-kube-api-access-8mfgz\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.344102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344016 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-etc-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344041 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-cnibin\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344065 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-k8s-cni-cncf-io\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344088 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-multus-certs\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344111 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-etc-kubernetes\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344137 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-ovn\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344164 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344187 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-hostroot\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344210 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nktql\" (UniqueName: \"kubernetes.io/projected/7e89584c-ddcf-48db-9695-696f9b47dea3-kube-api-access-nktql\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344239 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-socket-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344264 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-modprobe-d\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-kubernetes\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344311 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysctl-d\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-run-netns\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344361 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94xhv\" (UniqueName: \"kubernetes.io/projected/682756aa-6265-455c-8c8e-81e95817fd8a-kube-api-access-94xhv\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344385 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-socket-dir-parent\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4a6fb79-4e21-48a1-b3cd-630970335d30-host-slash\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.344553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344433 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-etc-selinux\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344461 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca72464b-98ac-4773-adec-7a0df6508fa7-tmp-dir\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344486 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzq8l\" (UniqueName: \"kubernetes.io/projected/b1a91258-14a4-45b3-ba6c-41ec9b31b731-kube-api-access-tzq8l\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344511 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-slash\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-ovnkube-config\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344564 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-var-lib-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344590 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344617 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-sys-fs\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344668 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b1a91258-14a4-45b3-ba6c-41ec9b31b731-serviceca\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344716 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-ovnkube-script-lib\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344741 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4a6fb79-4e21-48a1-b3cd-630970335d30-iptables-alerter-script\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344766 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-kubelet-config\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344791 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-lib-modules\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344816 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgz6s\" (UniqueName: \"kubernetes.io/projected/d4a6fb79-4e21-48a1-b3cd-630970335d30-kube-api-access-kgz6s\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344840 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysctl-conf\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.345070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-sys\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.345719 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344905 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzvwg\" (UniqueName: \"kubernetes.io/projected/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-kube-api-access-tzvwg\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.345719 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.344931 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-cni-bin\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.345719 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.345020 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-cni-bin\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.345719 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.345077 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ca72464b-98ac-4773-adec-7a0df6508fa7-hosts-file\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.345719 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.345356 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346090 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-ovn\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346134 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-sys-fs\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346141 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-kubelet-config\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346165 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-kubernetes\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346200 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4a6fb79-4e21-48a1-b3cd-630970335d30-host-slash\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346209 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-socket-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346220 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-run-netns\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346090 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346330 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-etc-selinux\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346383 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-device-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.346561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346464 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-dbus\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.347173 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346617 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca72464b-98ac-4773-adec-7a0df6508fa7-tmp-dir\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.347173 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346714 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.347173 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346851 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysctl-d\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.347173 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.346945 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-slash\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.347173 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347127 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-node-log\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.347414 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347186 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-kubelet\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.347414 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347261 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysctl-conf\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.347414 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347357 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-lib-modules\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.347414 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347366 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-env-overrides\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.347414 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347379 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.347647 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347419 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-cni-netd\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.347647 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347461 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-etc-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.347647 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347507 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-var-lib-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.347647 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347512 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-run\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.347647 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347579 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-modprobe-d\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.347647 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347615 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-openvswitch\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.347647 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347625 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b1a91258-14a4-45b3-ba6c-41ec9b31b731-host\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.348001 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347673 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.348001 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.347707 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:44.348001 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.347774 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret podName:ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:44.847753458 +0000 UTC m=+2.983043962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret") pod "global-pull-secret-syncer-nzcfc" (UID: "ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:44.348001 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.347988 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-systemd\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.348194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348025 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-systemd-units\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.348194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348109 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-ovnkube-config\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.348194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348159 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/682756aa-6265-455c-8c8e-81e95817fd8a-ovnkube-script-lib\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.348194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348160 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-system-cni-dir\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.348194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348180 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-sys\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.348423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348210 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cnibin\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.348423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348216 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9eab25ce-e60c-439c-8c4c-befdee9c1be3-cni-binary-copy\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.348423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348223 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d46d6ac8-8042-4cf4-921c-313657d78764-registration-dir\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.348423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348214 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-host-run-ovn-kubernetes\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.348423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348257 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-host\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.348423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348300 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-var-lib-kubelet\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.348423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348357 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.348746 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348558 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-sysconfig\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.348746 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348598 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-run-systemd\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.348746 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348725 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4a6fb79-4e21-48a1-b3cd-630970335d30-iptables-alerter-script\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.348975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348783 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9eab25ce-e60c-439c-8c4c-befdee9c1be3-os-release\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.348975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348819 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/682756aa-6265-455c-8c8e-81e95817fd8a-log-socket\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.348975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.348840 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/da1603f3-dd4b-4993-9730-75f00719033b-konnectivity-ca\") pod \"konnectivity-agent-qws5x\" (UID: \"da1603f3-dd4b-4993-9730-75f00719033b\") " pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:26:44.349538 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.349517 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-tmp\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.349737 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.349559 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b1a91258-14a4-45b3-ba6c-41ec9b31b731-serviceca\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.351242 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.351218 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/da1603f3-dd4b-4993-9730-75f00719033b-agent-certs\") pod \"konnectivity-agent-qws5x\" (UID: \"da1603f3-dd4b-4993-9730-75f00719033b\") " pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:26:44.351577 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.351537 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/682756aa-6265-455c-8c8e-81e95817fd8a-ovn-node-metrics-cert\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.351577 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.351552 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-etc-tuned\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.354476 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.354206 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:44.354476 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.354228 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:44.354476 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.354244 2574 projected.go:196] Error preparing data for projected volume kube-api-access-8dhlz for pod openshift-network-diagnostics/network-check-target-l68tb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:44.354476 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.354326 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz podName:6c1adf4b-17a9-4933-9038-63f5dfdcfd13 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:44.854308676 +0000 UTC m=+2.989599159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8dhlz" (UniqueName: "kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz") pod "network-check-target-l68tb" (UID: "6c1adf4b-17a9-4933-9038-63f5dfdcfd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:44.354749 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.354719 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jpcd\" (UniqueName: \"kubernetes.io/projected/ca72464b-98ac-4773-adec-7a0df6508fa7-kube-api-access-4jpcd\") pod \"node-resolver-6x7cb\" (UID: \"ca72464b-98ac-4773-adec-7a0df6508fa7\") " pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.354961 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.354921 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mfgz\" (UniqueName: \"kubernetes.io/projected/9eab25ce-e60c-439c-8c4c-befdee9c1be3-kube-api-access-8mfgz\") pod \"multus-additional-cni-plugins-lpd78\" (UID: \"9eab25ce-e60c-439c-8c4c-befdee9c1be3\") " pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.355063 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.355032 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94xhv\" (UniqueName: \"kubernetes.io/projected/682756aa-6265-455c-8c8e-81e95817fd8a-kube-api-access-94xhv\") pod \"ovnkube-node-9pbj9\" (UID: \"682756aa-6265-455c-8c8e-81e95817fd8a\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.357459 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.357420 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgz6s\" (UniqueName: \"kubernetes.io/projected/d4a6fb79-4e21-48a1-b3cd-630970335d30-kube-api-access-kgz6s\") pod \"iptables-alerter-hrz97\" (UID: \"d4a6fb79-4e21-48a1-b3cd-630970335d30\") " pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.357537 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.357505 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j888p\" (UniqueName: \"kubernetes.io/projected/d46d6ac8-8042-4cf4-921c-313657d78764-kube-api-access-j888p\") pod \"aws-ebs-csi-driver-node-rfss2\" (UID: \"d46d6ac8-8042-4cf4-921c-313657d78764\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.357537 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.357526 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzq8l\" (UniqueName: \"kubernetes.io/projected/b1a91258-14a4-45b3-ba6c-41ec9b31b731-kube-api-access-tzq8l\") pod \"node-ca-9hcgl\" (UID: \"b1a91258-14a4-45b3-ba6c-41ec9b31b731\") " pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.358878 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.358835 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzvwg\" (UniqueName: \"kubernetes.io/projected/ae70a485-c4b8-4469-bd47-b4002a8d6dbf-kube-api-access-tzvwg\") pod \"tuned-tztdg\" (UID: \"ae70a485-c4b8-4469-bd47-b4002a8d6dbf\") " pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.380074 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.380029 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" event={"ID":"4ae2047bf8edfd2708b684a75d4f25bd","Type":"ContainerStarted","Data":"62a8f90b1e8c6dc9dd90c13be052ec11abbf8092e02d6de6200c6966d3bfcb79"} Jul 06 01:26:44.381264 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.381214 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" event={"ID":"9ef1bea36b512041d79d06c57fe1854b","Type":"ContainerStarted","Data":"fc87b1851bcf9de77fe543e9ff7d82aba075c7dd4c13e1528ff30dac3d0b9826"} Jul 06 01:26:44.445361 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445326 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-cni-multus\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445500 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445366 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-daemon-config\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445500 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445443 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-cni-multus\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445627 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445500 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-cnibin\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445627 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445539 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-k8s-cni-cncf-io\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445627 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445564 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-multus-certs\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445627 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445586 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-etc-kubernetes\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445627 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445611 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-hostroot\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445880 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445633 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nktql\" (UniqueName: \"kubernetes.io/projected/7e89584c-ddcf-48db-9695-696f9b47dea3-kube-api-access-nktql\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445880 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445662 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-socket-dir-parent\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445880 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445713 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:44.445880 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445736 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8r5d\" (UniqueName: \"kubernetes.io/projected/b0a348e4-0203-4974-b406-002dc6a05d58-kube-api-access-r8r5d\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:44.445880 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-netns\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445880 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445790 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-cni-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.445880 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445846 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e89584c-ddcf-48db-9695-696f9b47dea3-cni-binary-copy\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445914 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-conf-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445935 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-daemon-config\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445943 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-system-cni-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.445969 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-cni-bin\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446002 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-os-release\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446011 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-socket-dir-parent\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446027 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-kubelet\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446062 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-cnibin\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446102 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-k8s-cni-cncf-io\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446104 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-kubelet\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446141 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-multus-certs\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446185 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-run-netns\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446224 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-hostroot\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446268 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-etc-kubernetes\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446429 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-system-cni-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446496 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-cni-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446496 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-host-var-lib-cni-bin\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446557 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-os-release\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.446845 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.446627 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:44.446845 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.446687 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:44.946668098 +0000 UTC m=+3.081958595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:44.447260 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446875 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e89584c-ddcf-48db-9695-696f9b47dea3-multus-conf-dir\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.447260 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.446894 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e89584c-ddcf-48db-9695-696f9b47dea3-cni-binary-copy\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.457415 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.457366 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8r5d\" (UniqueName: \"kubernetes.io/projected/b0a348e4-0203-4974-b406-002dc6a05d58-kube-api-access-r8r5d\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:44.457511 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.457430 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nktql\" (UniqueName: \"kubernetes.io/projected/7e89584c-ddcf-48db-9695-696f9b47dea3-kube-api-access-nktql\") pod \"multus-qts96\" (UID: \"7e89584c-ddcf-48db-9695-696f9b47dea3\") " pod="openshift-multus/multus-qts96" Jul 06 01:26:44.531370 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.531340 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hrz97" Jul 06 01:26:44.540498 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.540476 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" Jul 06 01:26:44.549365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.549343 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tztdg" Jul 06 01:26:44.555069 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.555052 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6x7cb" Jul 06 01:26:44.561727 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.561707 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9hcgl" Jul 06 01:26:44.570047 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.570028 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:26:44.578769 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.578747 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:26:44.585348 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.585330 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lpd78" Jul 06 01:26:44.589925 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.589908 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qts96" Jul 06 01:26:44.850882 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.850784 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:44.851035 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.851020 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:44.851112 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.851098 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret podName:ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:45.851077349 +0000 UTC m=+3.986367845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret") pod "global-pull-secret-syncer-nzcfc" (UID: "ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:44.952010 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.951972 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:44.952158 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:44.952035 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:44.952158 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.952083 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:44.952250 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.952160 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:45.952140025 +0000 UTC m=+4.087430531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:44.952250 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.952189 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:44.952250 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.952205 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:44.952250 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.952217 2574 projected.go:196] Error preparing data for projected volume kube-api-access-8dhlz for pod openshift-network-diagnostics/network-check-target-l68tb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:44.952436 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:44.952265 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz podName:6c1adf4b-17a9-4933-9038-63f5dfdcfd13 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:45.952248663 +0000 UTC m=+4.087539150 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dhlz" (UniqueName: "kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz") pod "network-check-target-l68tb" (UID: "6c1adf4b-17a9-4933-9038-63f5dfdcfd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:45.230510 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.230460 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd46d6ac8_8042_4cf4_921c_313657d78764.slice/crio-804fb72b212156ca9f44d4223c59f3a02e1240d67137f506dd471c0df54e90c2 WatchSource:0}: Error finding container 804fb72b212156ca9f44d4223c59f3a02e1240d67137f506dd471c0df54e90c2: Status 404 returned error can't find the container with id 804fb72b212156ca9f44d4223c59f3a02e1240d67137f506dd471c0df54e90c2 Jul 06 01:26:45.233812 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.233733 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae70a485_c4b8_4469_bd47_b4002a8d6dbf.slice/crio-76cc51deb5fa72cae89aaef88a923cdbb58f93cd3474b3cedc4e1f1769eb4787 WatchSource:0}: Error finding container 76cc51deb5fa72cae89aaef88a923cdbb58f93cd3474b3cedc4e1f1769eb4787: Status 404 returned error can't find the container with id 76cc51deb5fa72cae89aaef88a923cdbb58f93cd3474b3cedc4e1f1769eb4787 Jul 06 01:26:45.236179 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.236155 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e89584c_ddcf_48db_9695_696f9b47dea3.slice/crio-e38e8a81f632e630dc8c0b321c380d6629b5b64631250bf70b7258e1bca99242 WatchSource:0}: Error finding container e38e8a81f632e630dc8c0b321c380d6629b5b64631250bf70b7258e1bca99242: Status 404 returned error can't find the container with id e38e8a81f632e630dc8c0b321c380d6629b5b64631250bf70b7258e1bca99242 Jul 06 01:26:45.236828 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.236803 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod682756aa_6265_455c_8c8e_81e95817fd8a.slice/crio-b1344d7668eb14ba3b0d2568d9f22e2748cdad77f75bc91376aa66b056f225da WatchSource:0}: Error finding container b1344d7668eb14ba3b0d2568d9f22e2748cdad77f75bc91376aa66b056f225da: Status 404 returned error can't find the container with id b1344d7668eb14ba3b0d2568d9f22e2748cdad77f75bc91376aa66b056f225da Jul 06 01:26:45.239783 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.239758 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca72464b_98ac_4773_adec_7a0df6508fa7.slice/crio-ba15d33ddc534adac09c3a5f693d7e0cfb23cbf4ff53e8fb1be0c9a72cdcac33 WatchSource:0}: Error finding container ba15d33ddc534adac09c3a5f693d7e0cfb23cbf4ff53e8fb1be0c9a72cdcac33: Status 404 returned error can't find the container with id ba15d33ddc534adac09c3a5f693d7e0cfb23cbf4ff53e8fb1be0c9a72cdcac33 Jul 06 01:26:45.241198 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.240984 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a6fb79_4e21_48a1_b3cd_630970335d30.slice/crio-d75e10b7c4d882bc4e6ae21370bbeb452e392e2d9c4329869dc3f69890f7a1c3 WatchSource:0}: Error finding container d75e10b7c4d882bc4e6ae21370bbeb452e392e2d9c4329869dc3f69890f7a1c3: Status 404 returned error can't find the container with id d75e10b7c4d882bc4e6ae21370bbeb452e392e2d9c4329869dc3f69890f7a1c3 Jul 06 01:26:45.242005 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.241985 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1603f3_dd4b_4993_9730_75f00719033b.slice/crio-4260138cb3fcbcb5d9be7162ca0881007a859933eaf29352e8f4bb89dacd0b77 WatchSource:0}: Error finding container 4260138cb3fcbcb5d9be7162ca0881007a859933eaf29352e8f4bb89dacd0b77: Status 404 returned error can't find the container with id 4260138cb3fcbcb5d9be7162ca0881007a859933eaf29352e8f4bb89dacd0b77 Jul 06 01:26:45.242958 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.242932 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1a91258_14a4_45b3_ba6c_41ec9b31b731.slice/crio-a11de4d407b16f2bdf8e2505f37a513551a68942bad3eb38d3bb8dec8d57ac55 WatchSource:0}: Error finding container a11de4d407b16f2bdf8e2505f37a513551a68942bad3eb38d3bb8dec8d57ac55: Status 404 returned error can't find the container with id a11de4d407b16f2bdf8e2505f37a513551a68942bad3eb38d3bb8dec8d57ac55 Jul 06 01:26:45.243560 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:26:45.243538 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eab25ce_e60c_439c_8c4c_befdee9c1be3.slice/crio-f36baa4ebdcaf574af292917e450439943eb941f29e4eeecf9fea72ad0c40662 WatchSource:0}: Error finding container f36baa4ebdcaf574af292917e450439943eb941f29e4eeecf9fea72ad0c40662: Status 404 returned error can't find the container with id f36baa4ebdcaf574af292917e450439943eb941f29e4eeecf9fea72ad0c40662 Jul 06 01:26:45.267127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.267075 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 01:21:44 +0000 UTC" deadline="2027-12-09 03:55:13.048567011 +0000 UTC" Jul 06 01:26:45.267127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.267124 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12506h28m27.781447345s" Jul 06 01:26:45.385132 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.385098 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" event={"ID":"4ae2047bf8edfd2708b684a75d4f25bd","Type":"ContainerStarted","Data":"9121f155c6275956100580ba8725be63fc5d1ace6cbe799a2f3a425d5d6589b0"} Jul 06 01:26:45.386149 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.386125 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerStarted","Data":"f36baa4ebdcaf574af292917e450439943eb941f29e4eeecf9fea72ad0c40662"} Jul 06 01:26:45.387180 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.387160 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qts96" event={"ID":"7e89584c-ddcf-48db-9695-696f9b47dea3","Type":"ContainerStarted","Data":"e38e8a81f632e630dc8c0b321c380d6629b5b64631250bf70b7258e1bca99242"} Jul 06 01:26:45.388324 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.388305 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tztdg" event={"ID":"ae70a485-c4b8-4469-bd47-b4002a8d6dbf","Type":"ContainerStarted","Data":"76cc51deb5fa72cae89aaef88a923cdbb58f93cd3474b3cedc4e1f1769eb4787"} Jul 06 01:26:45.389386 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.389366 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" event={"ID":"d46d6ac8-8042-4cf4-921c-313657d78764","Type":"ContainerStarted","Data":"804fb72b212156ca9f44d4223c59f3a02e1240d67137f506dd471c0df54e90c2"} Jul 06 01:26:45.390365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.390347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9hcgl" event={"ID":"b1a91258-14a4-45b3-ba6c-41ec9b31b731","Type":"ContainerStarted","Data":"a11de4d407b16f2bdf8e2505f37a513551a68942bad3eb38d3bb8dec8d57ac55"} Jul 06 01:26:45.391336 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.391314 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qws5x" event={"ID":"da1603f3-dd4b-4993-9730-75f00719033b","Type":"ContainerStarted","Data":"4260138cb3fcbcb5d9be7162ca0881007a859933eaf29352e8f4bb89dacd0b77"} Jul 06 01:26:45.392530 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.392505 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hrz97" event={"ID":"d4a6fb79-4e21-48a1-b3cd-630970335d30","Type":"ContainerStarted","Data":"d75e10b7c4d882bc4e6ae21370bbeb452e392e2d9c4329869dc3f69890f7a1c3"} Jul 06 01:26:45.393548 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.393531 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6x7cb" event={"ID":"ca72464b-98ac-4773-adec-7a0df6508fa7","Type":"ContainerStarted","Data":"ba15d33ddc534adac09c3a5f693d7e0cfb23cbf4ff53e8fb1be0c9a72cdcac33"} Jul 06 01:26:45.394548 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.394530 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"b1344d7668eb14ba3b0d2568d9f22e2748cdad77f75bc91376aa66b056f225da"} Jul 06 01:26:45.397712 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.397672 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-193.ec2.internal" podStartSLOduration=2.397660194 podStartE2EDuration="2.397660194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:45.397288996 +0000 UTC m=+3.532579500" watchObservedRunningTime="2026-07-06 01:26:45.397660194 +0000 UTC m=+3.532950697" Jul 06 01:26:45.860014 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.859894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:45.860165 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:45.860040 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:45.860165 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:45.860106 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret podName:ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:47.860086562 +0000 UTC m=+5.995377044 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret") pod "global-pull-secret-syncer-nzcfc" (UID: "ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:45.961182 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.961140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:45.961355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:45.961217 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:45.961469 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:45.961358 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:45.961469 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:45.961375 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:45.961469 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:45.961388 2574 projected.go:196] Error preparing data for projected volume kube-api-access-8dhlz for pod openshift-network-diagnostics/network-check-target-l68tb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:45.961469 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:45.961443 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz podName:6c1adf4b-17a9-4933-9038-63f5dfdcfd13 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:47.96142546 +0000 UTC m=+6.096715964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dhlz" (UniqueName: "kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz") pod "network-check-target-l68tb" (UID: "6c1adf4b-17a9-4933-9038-63f5dfdcfd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:45.961796 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:45.961781 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:45.961881 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:45.961825 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:47.96181458 +0000 UTC m=+6.097105075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:46.214804 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:46.214725 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 01:26:46.268296 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:46.268240 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 01:21:44 +0000 UTC" deadline="2028-02-18 07:23:44.690532464 +0000 UTC" Jul 06 01:26:46.268296 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:46.268273 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14213h56m58.422263214s" Jul 06 01:26:46.376736 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:46.375970 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:46.376736 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:46.376101 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:26:46.376736 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:46.376492 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:46.376736 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:46.376581 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:26:46.376736 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:46.376639 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:46.376736 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:46.376699 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:26:46.414109 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:46.414071 2574 generic.go:356] "Generic (PLEG): container finished" podID="9ef1bea36b512041d79d06c57fe1854b" containerID="4304f30198ad8dfc7ce989b644793db25acb6d16157e3a769ede33f5d4e5a235" exitCode=0 Jul 06 01:26:46.414310 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:46.414155 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" event={"ID":"9ef1bea36b512041d79d06c57fe1854b","Type":"ContainerDied","Data":"4304f30198ad8dfc7ce989b644793db25acb6d16157e3a769ede33f5d4e5a235"} Jul 06 01:26:47.453687 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:47.453020 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" event={"ID":"9ef1bea36b512041d79d06c57fe1854b","Type":"ContainerStarted","Data":"943f5e168430436f0f368e03664fb66c05bb466a284c25995ef73066ff85680a"} Jul 06 01:26:47.880664 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:47.879954 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:47.880664 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:47.880117 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:47.880664 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:47.880177 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret podName:ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:51.88016024 +0000 UTC m=+10.015450745 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret") pod "global-pull-secret-syncer-nzcfc" (UID: "ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:47.982102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:47.981324 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:47.982102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:47.981386 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:47.982102 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:47.981520 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:47.982102 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:47.981538 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:47.982102 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:47.981552 2574 projected.go:196] Error preparing data for projected volume kube-api-access-8dhlz for pod openshift-network-diagnostics/network-check-target-l68tb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:47.982102 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:47.981609 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz podName:6c1adf4b-17a9-4933-9038-63f5dfdcfd13 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:51.981591296 +0000 UTC m=+10.116881800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dhlz" (UniqueName: "kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz") pod "network-check-target-l68tb" (UID: "6c1adf4b-17a9-4933-9038-63f5dfdcfd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:47.982102 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:47.982009 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:47.982102 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:47.982059 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:26:51.982042906 +0000 UTC m=+10.117333407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:48.377087 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:48.376981 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:48.377087 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:48.376993 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:48.377301 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:48.377109 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:26:48.377301 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:48.377181 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:26:48.377301 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:48.376994 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:48.377301 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:48.377288 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:26:50.376115 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:50.376081 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:50.376569 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:50.376130 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:50.376569 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:50.376081 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:50.376569 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:50.376232 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:26:50.376569 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:50.376329 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:26:50.376569 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:50.376418 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:26:51.917479 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:51.916882 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:51.917479 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:51.917057 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:51.917479 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:51.917118 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret podName:ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e nodeName:}" failed. No retries permitted until 2026-07-06 01:26:59.917100673 +0000 UTC m=+18.052391158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret") pod "global-pull-secret-syncer-nzcfc" (UID: "ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:52.019025 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:52.018241 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:52.019025 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:52.018314 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:52.019025 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.018441 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:26:52.019025 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.018455 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:26:52.019025 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.018464 2574 projected.go:196] Error preparing data for projected volume kube-api-access-8dhlz for pod openshift-network-diagnostics/network-check-target-l68tb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:52.019025 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.018519 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz podName:6c1adf4b-17a9-4933-9038-63f5dfdcfd13 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:00.018504957 +0000 UTC m=+18.153795440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dhlz" (UniqueName: "kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz") pod "network-check-target-l68tb" (UID: "6c1adf4b-17a9-4933-9038-63f5dfdcfd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:26:52.019025 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.018930 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:52.019025 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.018993 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:00.01897776 +0000 UTC m=+18.154268246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:26:52.377331 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:52.377254 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:52.377481 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.377431 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:26:52.377937 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:52.377916 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:52.378032 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.378012 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:26:52.378089 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:52.378071 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:52.378155 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:52.378129 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:26:54.377002 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:54.376966 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:54.377466 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:54.376972 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:54.377466 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:54.377105 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:26:54.377466 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:54.377152 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:26:54.377466 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:54.377170 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:54.377466 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:54.377257 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:26:56.375797 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:56.375762 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:56.376254 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:56.375810 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:56.376254 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:56.375762 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:56.376254 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:56.375908 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:26:56.376254 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:56.375988 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:26:56.376254 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:56.376077 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:26:58.376179 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:58.376146 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:58.376575 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:58.376241 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:26:58.376575 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:58.376300 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:26:58.376575 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:58.376389 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:26:58.376575 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:58.376456 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:26:58.376575 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:58.376507 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:26:59.987588 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:26:59.987504 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:26:59.987963 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:59.987640 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:26:59.987963 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:26:59.987697 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret podName:ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e nodeName:}" failed. No retries permitted until 2026-07-06 01:27:15.987682354 +0000 UTC m=+34.122972836 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret") pod "global-pull-secret-syncer-nzcfc" (UID: "ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:27:00.088409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:00.088368 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:00.088572 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:00.088433 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:00.088635 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.088578 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:27:00.088635 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.088603 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:27:00.088635 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.088616 2574 projected.go:196] Error preparing data for projected volume kube-api-access-8dhlz for pod openshift-network-diagnostics/network-check-target-l68tb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:27:00.088635 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.088618 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:27:00.088795 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.088681 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:16.088659805 +0000 UTC m=+34.223950291 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:27:00.088795 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.088700 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz podName:6c1adf4b-17a9-4933-9038-63f5dfdcfd13 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:16.088693063 +0000 UTC m=+34.223983560 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dhlz" (UniqueName: "kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz") pod "network-check-target-l68tb" (UID: "6c1adf4b-17a9-4933-9038-63f5dfdcfd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:27:00.376344 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:00.376263 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:00.376344 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:00.376263 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:00.376604 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:00.376396 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:00.376604 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.376396 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:00.376604 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.376473 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:00.376604 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:00.376543 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:02.376821 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:02.376792 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:02.377219 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:02.376883 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:02.377219 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:02.376891 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:02.377219 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:02.376913 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:02.377219 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:02.376953 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:02.377219 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:02.377078 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:04.376743 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.376050 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:04.376743 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:04.376182 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:04.376743 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.376599 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:04.376743 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:04.376701 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:04.377589 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.377444 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:04.377589 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:04.377544 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:04.486806 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.486652 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9hcgl" event={"ID":"b1a91258-14a4-45b3-ba6c-41ec9b31b731","Type":"ContainerStarted","Data":"ba672dc39b76bf627ed16e078fc5ec784c10f05f548238ac358cf8e42612a7c2"} Jul 06 01:27:04.488223 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.488199 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qws5x" event={"ID":"da1603f3-dd4b-4993-9730-75f00719033b","Type":"ContainerStarted","Data":"426237c4fc33461e5a16c37bbbdba1ac326d476460f87e0ae8ece72eb2e458ae"} Jul 06 01:27:04.489450 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.489406 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6x7cb" event={"ID":"ca72464b-98ac-4773-adec-7a0df6508fa7","Type":"ContainerStarted","Data":"759fe921240b287becd3d89918601c9dbf5e4bf841f395cfec909f867e8f799d"} Jul 06 01:27:04.490775 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.490754 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"e1686cc3849d5169c01d5cbf2ff8094dd5741e8faf7566f5e6d6cb16262f9fac"} Jul 06 01:27:04.492279 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.492258 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerStarted","Data":"1c1f090d5700cfa330ae9ed386b755badb38cb748f69f1387a46319ad125221a"} Jul 06 01:27:04.493779 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.493762 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tztdg" event={"ID":"ae70a485-c4b8-4469-bd47-b4002a8d6dbf","Type":"ContainerStarted","Data":"485bf197e600f9627e720aed12c64a54bbfe5c09dbb26fae85a0fd5615c83a11"} Jul 06 01:27:04.494967 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.494946 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" event={"ID":"d46d6ac8-8042-4cf4-921c-313657d78764","Type":"ContainerStarted","Data":"ff2efd253fc29c7561a1a96a911e0d74fd43fb8be704150fa6121e838d050046"} Jul 06 01:27:04.512140 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.511980 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-193.ec2.internal" podStartSLOduration=21.511966837 podStartE2EDuration="21.511966837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:26:47.470588159 +0000 UTC m=+5.605878664" watchObservedRunningTime="2026-07-06 01:27:04.511966837 +0000 UTC m=+22.647257342" Jul 06 01:27:04.527317 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.527276 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6x7cb" podStartSLOduration=3.515797316 podStartE2EDuration="22.527260525s" totalImagesPullingTime="19.011463209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:04.527137918 +0000 UTC m=+22.662428422" watchObservedRunningTime="2026-07-06 01:27:04.527260525 +0000 UTC m=+22.662551031" Jul 06 01:27:04.539989 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.539956 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9hcgl" podStartSLOduration=3.5316921839999997 podStartE2EDuration="22.539943771s" totalImagesPullingTime="19.008251587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:04.539639866 +0000 UTC m=+22.674930370" watchObservedRunningTime="2026-07-06 01:27:04.539943771 +0000 UTC m=+22.675234275" Jul 06 01:27:04.552674 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.552632 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qws5x" podStartSLOduration=8.266730386 podStartE2EDuration="22.55262187s" totalImagesPullingTime="14.285891484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:04.552464739 +0000 UTC m=+22.687755243" watchObservedRunningTime="2026-07-06 01:27:04.55262187 +0000 UTC m=+22.687912374" Jul 06 01:27:04.570374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:04.570326 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tztdg" podStartSLOduration=3.551812771 podStartE2EDuration="22.570306234s" totalImagesPullingTime="19.018493463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:04.569250431 +0000 UTC m=+22.704540927" watchObservedRunningTime="2026-07-06 01:27:04.570306234 +0000 UTC m=+22.705596738" Jul 06 01:27:05.499622 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.499451 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:27:05.500409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.499934 2574 generic.go:356] "Generic (PLEG): container finished" podID="682756aa-6265-455c-8c8e-81e95817fd8a" containerID="298d94eba7a5886088051321595ece3ce5b5fc614fbae544d7029942dc7636ed" exitCode=1 Jul 06 01:27:05.500409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.500001 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"a2cd684ce5a708bf3034f0e8429472c5a3ce10248ec7aa94287243b7194395f3"} Jul 06 01:27:05.500409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.500110 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"68faaf248ee1a983b876d1da594a85cbaf74f81bb79878a431ac35150b26f8d5"} Jul 06 01:27:05.500409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.500148 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"a00a077938a2be9092c2e35997545f86bea4fee2fd71df61b10b2cab2107a897"} Jul 06 01:27:05.500409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.500175 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"0c21eff20ca2c08c5ec3fc157e08608bac843f0e871d7881c81d6f80f4e86ea6"} Jul 06 01:27:05.500409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.500218 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerDied","Data":"298d94eba7a5886088051321595ece3ce5b5fc614fbae544d7029942dc7636ed"} Jul 06 01:27:05.501335 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.501315 2574 generic.go:356] "Generic (PLEG): container finished" podID="9eab25ce-e60c-439c-8c4c-befdee9c1be3" containerID="1c1f090d5700cfa330ae9ed386b755badb38cb748f69f1387a46319ad125221a" exitCode=0 Jul 06 01:27:05.501420 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.501372 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerDied","Data":"1c1f090d5700cfa330ae9ed386b755badb38cb748f69f1387a46319ad125221a"} Jul 06 01:27:05.502875 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.502797 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qts96" event={"ID":"7e89584c-ddcf-48db-9695-696f9b47dea3","Type":"ContainerStarted","Data":"120da81179c93763a4c89ddd146b423272370c69be94c233a50e4fa8a28ad3db"} Jul 06 01:27:05.538992 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.538954 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qts96" podStartSLOduration=4.201709303 podStartE2EDuration="23.538942172s" totalImagesPullingTime="19.337232869s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:05.538927692 +0000 UTC m=+23.674218197" watchObservedRunningTime="2026-07-06 01:27:05.538942172 +0000 UTC m=+23.674232654" Jul 06 01:27:05.959228 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:05.959208 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 01:27:06.305396 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.305308 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T01:27:05.959224711Z","UUID":"069d08bd-75a1-4c10-8269-695a0001a3f2","Handler":null,"Name":"","Endpoint":""} Jul 06 01:27:06.306629 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.306606 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 01:27:06.306742 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.306639 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 01:27:06.376223 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.376197 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:06.376377 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:06.376323 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:06.376377 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.376196 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:06.376377 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.376347 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:06.376537 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:06.376404 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:06.376537 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:06.376472 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:06.508247 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.508208 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" event={"ID":"d46d6ac8-8042-4cf4-921c-313657d78764","Type":"ContainerStarted","Data":"a41ff96a15924670424628e9f9342a3b424a6a0890a6cdee7a1d8cf7af69aaa8"} Jul 06 01:27:06.510430 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.510404 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hrz97" event={"ID":"d4a6fb79-4e21-48a1-b3cd-630970335d30","Type":"ContainerStarted","Data":"50a0d9c7d824085877cf7080fea8376c1966cf52ff47b136f924851a3ae94be7"} Jul 06 01:27:06.524056 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.524003 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hrz97" podStartSLOduration=10.23707879 podStartE2EDuration="24.523984478s" totalImagesPullingTime="14.286905688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:06.523075397 +0000 UTC m=+24.658365902" watchObservedRunningTime="2026-07-06 01:27:06.523984478 +0000 UTC m=+24.659274983" Jul 06 01:27:06.662388 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.662308 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qws5x_da1603f3-dd4b-4993-9730-75f00719033b/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:06.770417 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:06.770385 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qws5x_da1603f3-dd4b-4993-9730-75f00719033b/konnectivity-agent/0.log" limit=1024 Jul 06 01:27:07.514392 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:07.514213 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:27:07.514791 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:07.514695 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"c6b93cfda23881a94d293a485e059c183712ca9d768e02595fa538d2e34209e9"} Jul 06 01:27:07.516543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:07.516514 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" event={"ID":"d46d6ac8-8042-4cf4-921c-313657d78764","Type":"ContainerStarted","Data":"f9e14eab894896f4788293205c316bc42b5c3fecfc67919828709f3124537cc8"} Jul 06 01:27:07.534438 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:07.534318 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rfss2" podStartSLOduration=3.980492688 podStartE2EDuration="25.5342976s" totalImagesPullingTime="21.553804912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:07.53395909 +0000 UTC m=+25.669249594" watchObservedRunningTime="2026-07-06 01:27:07.5342976 +0000 UTC m=+25.669588105" Jul 06 01:27:08.375922 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:08.375895 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:08.376211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:08.375900 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:08.376211 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:08.376008 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:08.376211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:08.376047 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:08.376211 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:08.376126 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:08.376211 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:08.376207 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:09.174846 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:09.174811 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:27:09.175509 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:09.175410 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:27:09.520701 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:09.520628 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:27:09.521180 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:09.521162 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qws5x" Jul 06 01:27:10.376411 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:10.376382 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:10.376914 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:10.376382 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:10.376914 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:10.376503 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:10.376914 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:10.376557 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:10.376914 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:10.376660 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:10.376914 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:10.376728 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:11.526383 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:11.526170 2574 generic.go:356] "Generic (PLEG): container finished" podID="9eab25ce-e60c-439c-8c4c-befdee9c1be3" containerID="2066bfb2ece6c23f75f3ac41717ec719ed0d3bb703d82ca866e9c16838740afc" exitCode=0 Jul 06 01:27:11.527179 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:11.526249 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerDied","Data":"2066bfb2ece6c23f75f3ac41717ec719ed0d3bb703d82ca866e9c16838740afc"} Jul 06 01:27:11.529807 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:11.529719 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:27:11.530107 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:11.530089 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"40b05f191a21ca6276ef204b9e2344883e3ec7a0f66ec5c09a7fd53e4b65f219"} Jul 06 01:27:11.530508 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:11.530495 2574 scope.go:117] "RemoveContainer" containerID="298d94eba7a5886088051321595ece3ce5b5fc614fbae544d7029942dc7636ed" Jul 06 01:27:12.376903 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.376876 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:12.377044 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:12.376982 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:12.377170 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.377150 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:12.377265 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:12.377248 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:12.377313 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.377276 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:12.377352 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:12.377325 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:12.536151 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.535998 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:27:12.536666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.536642 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" event={"ID":"682756aa-6265-455c-8c8e-81e95817fd8a","Type":"ContainerStarted","Data":"5cc5635cf60cbae7cdd5373ebaabda2b4cee4cfa89ec6873022fd27faac864c2"} Jul 06 01:27:12.536936 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.536913 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:27:12.537117 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.537094 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:27:12.537213 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.537166 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:27:12.552425 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.552406 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:27:12.552570 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.552557 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:27:12.563123 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.563086 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" podStartSLOduration=11.499506788 podStartE2EDuration="30.563074718s" totalImagesPullingTime="19.06356793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:12.562705573 +0000 UTC m=+30.697996077" watchObservedRunningTime="2026-07-06 01:27:12.563074718 +0000 UTC m=+30.698365221" Jul 06 01:27:12.595273 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.595230 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzcfc"] Jul 06 01:27:12.595420 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.595329 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:12.595486 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:12.595425 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:12.595835 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.595802 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l68tb"] Jul 06 01:27:12.595929 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.595896 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:12.596011 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:12.595990 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:12.596366 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.596352 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cn8h2"] Jul 06 01:27:12.596421 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:12.596412 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:12.596506 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:12.596491 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:13.540948 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:13.540917 2574 generic.go:356] "Generic (PLEG): container finished" podID="9eab25ce-e60c-439c-8c4c-befdee9c1be3" containerID="2296e1a738dd03b79ce0ae0364fba799a5301f0b5d174bbd4f417cbace5b20fe" exitCode=0 Jul 06 01:27:13.541435 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:13.540998 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerDied","Data":"2296e1a738dd03b79ce0ae0364fba799a5301f0b5d174bbd4f417cbace5b20fe"} Jul 06 01:27:14.376280 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:14.376247 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:14.376426 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:14.376251 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:14.376426 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:14.376343 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:14.376500 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:14.376412 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:14.376500 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:14.376486 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:14.376571 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:14.376555 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:15.546715 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:15.546674 2574 generic.go:356] "Generic (PLEG): container finished" podID="9eab25ce-e60c-439c-8c4c-befdee9c1be3" containerID="d7c0c725a2a3ec0fcf52d53e57980dc2a41f099bacec2397c0c361e0cfe5ce3f" exitCode=0 Jul 06 01:27:15.547310 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:15.546723 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerDied","Data":"d7c0c725a2a3ec0fcf52d53e57980dc2a41f099bacec2397c0c361e0cfe5ce3f"} Jul 06 01:27:16.004022 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:16.003986 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:16.004147 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.004125 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 01:27:16.004185 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.004180 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret podName:ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e nodeName:}" failed. No retries permitted until 2026-07-06 01:27:48.004165555 +0000 UTC m=+66.139456037 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret") pod "global-pull-secret-syncer-nzcfc" (UID: "ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e") : object "kube-system"/"original-pull-secret" not registered Jul 06 01:27:16.105325 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:16.105148 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:16.105457 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:16.105353 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:16.105457 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.105294 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:27:16.105457 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.105441 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:48.105420846 +0000 UTC m=+66.240711351 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 01:27:16.105611 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.105488 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 01:27:16.105611 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.105500 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 01:27:16.105611 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.105509 2574 projected.go:196] Error preparing data for projected volume kube-api-access-8dhlz for pod openshift-network-diagnostics/network-check-target-l68tb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:27:16.105611 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.105556 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz podName:6c1adf4b-17a9-4933-9038-63f5dfdcfd13 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:48.105545908 +0000 UTC m=+66.240836403 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8dhlz" (UniqueName: "kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz") pod "network-check-target-l68tb" (UID: "6c1adf4b-17a9-4933-9038-63f5dfdcfd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 01:27:16.375877 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:16.375840 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:16.376027 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.375943 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l68tb" podUID="6c1adf4b-17a9-4933-9038-63f5dfdcfd13" Jul 06 01:27:16.376027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:16.375978 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:16.376027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:16.375841 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:16.376120 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.376068 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:27:16.376157 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:16.376124 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcfc" podUID="ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e" Jul 06 01:27:18.157370 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.157339 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-193.ec2.internal" event="NodeReady" Jul 06 01:27:18.157803 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.157439 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 01:27:18.232556 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.232437 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4q7d7"] Jul 06 01:27:18.234173 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.234152 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4q7d7" Jul 06 01:27:18.238190 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.238162 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 01:27:18.238330 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.238281 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bszxs\"" Jul 06 01:27:18.238543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.238524 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 01:27:18.238621 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.238557 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 01:27:18.239714 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.239696 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dmkmq"] Jul 06 01:27:18.242161 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.242136 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.245573 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.245512 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 01:27:18.245777 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.245757 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 01:27:18.246227 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.246028 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-58ndz\"" Jul 06 01:27:18.247285 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.247265 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4q7d7"] Jul 06 01:27:18.259306 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.259284 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dmkmq"] Jul 06 01:27:18.375967 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.375938 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:18.375967 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.375956 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:18.376190 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.375940 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:18.379138 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.379120 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 01:27:18.379289 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.379185 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 01:27:18.379442 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.379416 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n24x9\"" Jul 06 01:27:18.379442 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.379428 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qszpf\"" Jul 06 01:27:18.379604 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.379526 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 01:27:18.379658 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.379646 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 01:27:18.421062 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.421034 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlnlc\" (UniqueName: \"kubernetes.io/projected/2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832-kube-api-access-xlnlc\") pod \"ingress-canary-4q7d7\" (UID: \"2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832\") " pod="openshift-ingress-canary/ingress-canary-4q7d7" Jul 06 01:27:18.421174 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.421095 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b07fe06c-ab11-4f51-8a99-286d02bad557-tmp-dir\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.421174 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.421119 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832-cert\") pod \"ingress-canary-4q7d7\" (UID: \"2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832\") " pod="openshift-ingress-canary/ingress-canary-4q7d7" Jul 06 01:27:18.421282 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.421224 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.421340 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.421287 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wpg6\" (UniqueName: \"kubernetes.io/projected/b07fe06c-ab11-4f51-8a99-286d02bad557-kube-api-access-2wpg6\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.421379 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.421348 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b07fe06c-ab11-4f51-8a99-286d02bad557-config-volume\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.522533 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.522459 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b07fe06c-ab11-4f51-8a99-286d02bad557-config-volume\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.522533 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.522512 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlnlc\" (UniqueName: \"kubernetes.io/projected/2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832-kube-api-access-xlnlc\") pod \"ingress-canary-4q7d7\" (UID: \"2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832\") " pod="openshift-ingress-canary/ingress-canary-4q7d7" Jul 06 01:27:18.522755 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.522583 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b07fe06c-ab11-4f51-8a99-286d02bad557-tmp-dir\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.522755 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.522609 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832-cert\") pod \"ingress-canary-4q7d7\" (UID: \"2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832\") " pod="openshift-ingress-canary/ingress-canary-4q7d7" Jul 06 01:27:18.522901 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.522810 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.522901 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.522844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wpg6\" (UniqueName: \"kubernetes.io/projected/b07fe06c-ab11-4f51-8a99-286d02bad557-kube-api-access-2wpg6\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.523008 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.522957 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b07fe06c-ab11-4f51-8a99-286d02bad557-tmp-dir\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.523008 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:18.522974 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:18.523115 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:18.523044 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:19.02302298 +0000 UTC m=+37.158313466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:27:18.523180 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.523163 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b07fe06c-ab11-4f51-8a99-286d02bad557-config-volume\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.527481 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.527459 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832-cert\") pod \"ingress-canary-4q7d7\" (UID: \"2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832\") " pod="openshift-ingress-canary/ingress-canary-4q7d7" Jul 06 01:27:18.532976 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.532957 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlnlc\" (UniqueName: \"kubernetes.io/projected/2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832-kube-api-access-xlnlc\") pod \"ingress-canary-4q7d7\" (UID: \"2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832\") " pod="openshift-ingress-canary/ingress-canary-4q7d7" Jul 06 01:27:18.534536 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.534509 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wpg6\" (UniqueName: \"kubernetes.io/projected/b07fe06c-ab11-4f51-8a99-286d02bad557-kube-api-access-2wpg6\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:18.548185 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.548159 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4q7d7" Jul 06 01:27:18.692594 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:18.692559 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4q7d7"] Jul 06 01:27:18.697649 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:27:18.697623 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e0ee386_6ef9_4c02_8d0d_5cb81a1b2832.slice/crio-ddac5f9655ea1df27939c96c5700cdfba44fe00e39e8d0913b8022f9c964e481 WatchSource:0}: Error finding container ddac5f9655ea1df27939c96c5700cdfba44fe00e39e8d0913b8022f9c964e481: Status 404 returned error can't find the container with id ddac5f9655ea1df27939c96c5700cdfba44fe00e39e8d0913b8022f9c964e481 Jul 06 01:27:19.026447 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:19.026416 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:19.026633 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:19.026578 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:19.026706 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:19.026656 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:20.026635223 +0000 UTC m=+38.161925711 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:27:19.555674 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:19.555639 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4q7d7" event={"ID":"2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832","Type":"ContainerStarted","Data":"ddac5f9655ea1df27939c96c5700cdfba44fe00e39e8d0913b8022f9c964e481"} Jul 06 01:27:20.033531 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:20.033498 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:20.033693 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:20.033664 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:20.033740 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:20.033734 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:22.033718282 +0000 UTC m=+40.169008763 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:27:22.050342 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:22.050260 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:22.050668 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:22.050410 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:22.050668 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:22.050509 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:26.050492586 +0000 UTC m=+44.185783090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:27:22.564619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:22.564585 2574 generic.go:356] "Generic (PLEG): container finished" podID="9eab25ce-e60c-439c-8c4c-befdee9c1be3" containerID="16f3e281682ea9c50a3720a21f088d868463186132a3984ad4c405b5c7b80e4d" exitCode=0 Jul 06 01:27:22.564777 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:22.564657 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerDied","Data":"16f3e281682ea9c50a3720a21f088d868463186132a3984ad4c405b5c7b80e4d"} Jul 06 01:27:22.566021 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:22.565998 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4q7d7" event={"ID":"2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832","Type":"ContainerStarted","Data":"8576bcb3cb02ab3f744854b2ef9726ecefd232c0192780f4747e11f8ed0e9022"} Jul 06 01:27:22.616133 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:22.616068 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4q7d7" podStartSLOduration=1.664536466 podStartE2EDuration="4.616045797s" totalImagesPullingTime="2.951509331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:22.614804268 +0000 UTC m=+40.750094773" watchObservedRunningTime="2026-07-06 01:27:22.616045797 +0000 UTC m=+40.751336302" Jul 06 01:27:23.571499 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:23.571458 2574 generic.go:356] "Generic (PLEG): container finished" podID="9eab25ce-e60c-439c-8c4c-befdee9c1be3" containerID="d6b217a78ff7dc69e952d268c2604b30edb9a8cca11222c31bbac750eaebb680" exitCode=0 Jul 06 01:27:23.571884 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:23.571526 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerDied","Data":"d6b217a78ff7dc69e952d268c2604b30edb9a8cca11222c31bbac750eaebb680"} Jul 06 01:27:24.577318 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:24.577278 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lpd78" event={"ID":"9eab25ce-e60c-439c-8c4c-befdee9c1be3","Type":"ContainerStarted","Data":"3b5a778a3759edb151676e7af0b293d0c58557fff20305f7a4c737ea81f52104"} Jul 06 01:27:24.605756 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:24.605706 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lpd78" podStartSLOduration=9.888879131 podStartE2EDuration="42.605688921s" totalImagesPullingTime="32.71680979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:24.605661651 +0000 UTC m=+42.740952157" watchObservedRunningTime="2026-07-06 01:27:24.605688921 +0000 UTC m=+42.740979425" Jul 06 01:27:26.077166 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:26.077131 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:26.077623 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:26.077299 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:26.077623 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:26.077379 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:34.077356094 +0000 UTC m=+52.212646593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:27:28.201397 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.201367 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn"] Jul 06 01:27:28.223348 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.223318 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn"] Jul 06 01:27:28.223694 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.223386 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:27:28.227085 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.226989 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 01:27:28.227188 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.227133 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 01:27:28.227525 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.227475 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 01:27:28.227703 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.226992 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-225zm\"" Jul 06 01:27:28.228364 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.228342 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 01:27:28.294032 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.294006 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f6ef509-306b-4242-b343-63a0bbf92566-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75bdb96784-f2dxn\" (UID: \"9f6ef509-306b-4242-b343-63a0bbf92566\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:27:28.294145 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.294077 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6j4k\" (UniqueName: \"kubernetes.io/projected/9f6ef509-306b-4242-b343-63a0bbf92566-kube-api-access-w6j4k\") pod \"managed-serviceaccount-addon-agent-75bdb96784-f2dxn\" (UID: \"9f6ef509-306b-4242-b343-63a0bbf92566\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:27:28.394490 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.394462 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f6ef509-306b-4242-b343-63a0bbf92566-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75bdb96784-f2dxn\" (UID: \"9f6ef509-306b-4242-b343-63a0bbf92566\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:27:28.394605 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.394496 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6j4k\" (UniqueName: \"kubernetes.io/projected/9f6ef509-306b-4242-b343-63a0bbf92566-kube-api-access-w6j4k\") pod \"managed-serviceaccount-addon-agent-75bdb96784-f2dxn\" (UID: \"9f6ef509-306b-4242-b343-63a0bbf92566\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:27:28.397585 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.397566 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9f6ef509-306b-4242-b343-63a0bbf92566-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-75bdb96784-f2dxn\" (UID: \"9f6ef509-306b-4242-b343-63a0bbf92566\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:27:28.402584 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.402563 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6j4k\" (UniqueName: \"kubernetes.io/projected/9f6ef509-306b-4242-b343-63a0bbf92566-kube-api-access-w6j4k\") pod \"managed-serviceaccount-addon-agent-75bdb96784-f2dxn\" (UID: \"9f6ef509-306b-4242-b343-63a0bbf92566\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:27:28.549387 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.549330 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:27:28.701344 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:28.701311 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn"] Jul 06 01:27:28.704569 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:27:28.704543 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f6ef509_306b_4242_b343_63a0bbf92566.slice/crio-b5299d456cb8be179d867f406bba4cf11642ecedfa3f9297adedf75df02c5077 WatchSource:0}: Error finding container b5299d456cb8be179d867f406bba4cf11642ecedfa3f9297adedf75df02c5077: Status 404 returned error can't find the container with id b5299d456cb8be179d867f406bba4cf11642ecedfa3f9297adedf75df02c5077 Jul 06 01:27:29.589469 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:29.589441 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" event={"ID":"9f6ef509-306b-4242-b343-63a0bbf92566","Type":"ContainerStarted","Data":"b5299d456cb8be179d867f406bba4cf11642ecedfa3f9297adedf75df02c5077"} Jul 06 01:27:32.597929 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:32.597893 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" event={"ID":"9f6ef509-306b-4242-b343-63a0bbf92566","Type":"ContainerStarted","Data":"415c9834317ed6d3bdbe878b4145218e92a6b3974083a6d46c0df1e6b1f21a3d"} Jul 06 01:27:32.613754 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:32.613713 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" podStartSLOduration=1.294203041 podStartE2EDuration="4.613698347s" totalImagesPullingTime="3.319495306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:32.612964927 +0000 UTC m=+50.748255433" watchObservedRunningTime="2026-07-06 01:27:32.613698347 +0000 UTC m=+50.748988851" Jul 06 01:27:34.137294 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:34.137263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:34.137686 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:34.137387 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:34.137686 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:34.137445 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:27:50.137432214 +0000 UTC m=+68.272722695 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:27:44.553398 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:44.553373 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pbj9" Jul 06 01:27:48.025768 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.025733 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:48.028211 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.028191 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 01:27:48.038147 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.038124 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e-original-pull-secret\") pod \"global-pull-secret-syncer-nzcfc\" (UID: \"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e\") " pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:48.089070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.089040 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcfc" Jul 06 01:27:48.126130 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.126096 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:27:48.126296 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.126152 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:48.129512 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.129487 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 01:27:48.129614 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.129572 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 01:27:48.146111 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:48.137085 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 01:27:48.146111 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:48.137179 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:28:52.137154781 +0000 UTC m=+130.272445280 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : secret "metrics-daemon-secret" not found Jul 06 01:27:48.146111 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.144008 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 01:27:48.160979 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.160953 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhlz\" (UniqueName: \"kubernetes.io/projected/6c1adf4b-17a9-4933-9038-63f5dfdcfd13-kube-api-access-8dhlz\") pod \"network-check-target-l68tb\" (UID: \"6c1adf4b-17a9-4933-9038-63f5dfdcfd13\") " pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:48.231920 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.231892 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzcfc"] Jul 06 01:27:48.402814 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.402793 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n24x9\"" Jul 06 01:27:48.411428 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.411410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:48.526136 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.526101 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l68tb"] Jul 06 01:27:48.530596 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:27:48.530569 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1adf4b_17a9_4933_9038_63f5dfdcfd13.slice/crio-816bb75261a6637d37d1ff19a289245ec251d185d2c33c03670a52b5fb7ccc2d WatchSource:0}: Error finding container 816bb75261a6637d37d1ff19a289245ec251d185d2c33c03670a52b5fb7ccc2d: Status 404 returned error can't find the container with id 816bb75261a6637d37d1ff19a289245ec251d185d2c33c03670a52b5fb7ccc2d Jul 06 01:27:48.635381 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.635342 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzcfc" event={"ID":"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e","Type":"ContainerStarted","Data":"1c70b66d66733b2e5428639d32831045f6cfb3e1589716c6bdcbe98fcafb2a09"} Jul 06 01:27:48.636416 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:48.636395 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l68tb" event={"ID":"6c1adf4b-17a9-4933-9038-63f5dfdcfd13","Type":"ContainerStarted","Data":"816bb75261a6637d37d1ff19a289245ec251d185d2c33c03670a52b5fb7ccc2d"} Jul 06 01:27:50.140451 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:50.140388 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:27:50.140846 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:50.140533 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:27:50.140846 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:27:50.140633 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:28:22.140586287 +0000 UTC m=+100.275876775 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:27:53.649678 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:53.649635 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzcfc" event={"ID":"ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e","Type":"ContainerStarted","Data":"8fa03cc74877246b7ec6e87d8d185dfc7f88f03b461060cf2c14a5a507b92247"} Jul 06 01:27:53.651069 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:53.651044 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l68tb" event={"ID":"6c1adf4b-17a9-4933-9038-63f5dfdcfd13","Type":"ContainerStarted","Data":"2edae4ea3934200165e282e451063135f9dfa1753b4aa46cbc9f5093c0e6401f"} Jul 06 01:27:53.651170 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:53.651099 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:27:53.666584 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:53.666542 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nzcfc" podStartSLOduration=66.920365721 podStartE2EDuration="1m11.66652954s" totalImagesPullingTime="4.746163819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:53.665709795 +0000 UTC m=+71.801000302" watchObservedRunningTime="2026-07-06 01:27:53.66652954 +0000 UTC m=+71.801820043" Jul 06 01:27:53.681532 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:27:53.681496 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l68tb" podStartSLOduration=67.230576783 podStartE2EDuration="1m11.681488128s" totalImagesPullingTime="4.450911345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:27:53.681167925 +0000 UTC m=+71.816458425" watchObservedRunningTime="2026-07-06 01:27:53.681488128 +0000 UTC m=+71.816778632" Jul 06 01:28:22.145728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:28:22.145604 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:28:22.145728 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:28:22.145723 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:28:22.146292 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:28:22.145788 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:29:26.145772576 +0000 UTC m=+164.281063060 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:28:24.654875 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:28:24.654833 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l68tb" Jul 06 01:28:52.234285 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:28:52.234252 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:28:52.234746 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:28:52.234358 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 01:28:52.234746 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:28:52.234421 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:30:54.23439961 +0000 UTC m=+252.369690092 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : secret "metrics-daemon-secret" not found Jul 06 01:29:21.255888 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:29:21.255831 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dmkmq" podUID="b07fe06c-ab11-4f51-8a99-286d02bad557" Jul 06 01:29:21.394391 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:29:21.394363 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:29:21.861181 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:29:21.861153 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dmkmq" Jul 06 01:29:26.161581 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:29:26.161544 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:29:26.161997 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:29:26.161661 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 01:29:26.161997 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:29:26.161715 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls podName:b07fe06c-ab11-4f51-8a99-286d02bad557 nodeName:}" failed. No retries permitted until 2026-07-06 01:31:28.161699984 +0000 UTC m=+286.296990467 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls") pod "dns-default-dmkmq" (UID: "b07fe06c-ab11-4f51-8a99-286d02bad557") : secret "dns-default-metrics-tls" not found Jul 06 01:29:32.887197 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:29:32.887162 2574 generic.go:356] "Generic (PLEG): container finished" podID="9f6ef509-306b-4242-b343-63a0bbf92566" containerID="415c9834317ed6d3bdbe878b4145218e92a6b3974083a6d46c0df1e6b1f21a3d" exitCode=255 Jul 06 01:29:32.887585 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:29:32.887224 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" event={"ID":"9f6ef509-306b-4242-b343-63a0bbf92566","Type":"ContainerDied","Data":"415c9834317ed6d3bdbe878b4145218e92a6b3974083a6d46c0df1e6b1f21a3d"} Jul 06 01:29:32.892695 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:29:32.892677 2574 scope.go:117] "RemoveContainer" containerID="415c9834317ed6d3bdbe878b4145218e92a6b3974083a6d46c0df1e6b1f21a3d" Jul 06 01:29:33.896188 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:29:33.896115 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" event={"ID":"9f6ef509-306b-4242-b343-63a0bbf92566","Type":"ContainerStarted","Data":"2ab85e2d065de1b3e271252ce638565bab5c042f010d369f722ca332fcf11fec"} Jul 06 01:29:34.376264 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:29:34.376238 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:30:54.213103 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.213073 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7995fbff9c-klmcz"] Jul 06 01:30:54.214649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.214635 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65c5cc8998-9s4wf"] Jul 06 01:30:54.214768 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.214750 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.216232 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.216217 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.216779 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.216762 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gbpkl\"" Jul 06 01:30:54.216849 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.216765 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 01:30:54.217025 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.217004 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 01:30:54.217139 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.217033 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 01:30:54.217139 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.217007 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 01:30:54.217307 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.217293 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 01:30:54.217891 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.217877 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 01:30:54.218929 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.218915 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 01:30:54.219265 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.219252 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 01:30:54.219967 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.219954 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qzthl\"" Jul 06 01:30:54.220136 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.220123 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 01:30:54.227194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.227175 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 01:30:54.235393 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.235373 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7995fbff9c-klmcz"] Jul 06 01:30:54.237351 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.237332 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65c5cc8998-9s4wf"] Jul 06 01:30:54.305172 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.305150 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-vtffd"] Jul 06 01:30:54.306685 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.306671 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.308459 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.308440 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-hvddn"] Jul 06 01:30:54.309221 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.309206 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 01:30:54.309496 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.309480 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 01:30:54.309901 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.309888 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.310147 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.310126 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hsdbn\"" Jul 06 01:30:54.310234 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.310161 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 01:30:54.310234 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.310185 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 01:30:54.311662 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.311645 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 01:30:54.311759 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.311714 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zfx7f\"" Jul 06 01:30:54.312091 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.312060 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:30:54.312188 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.312139 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 01:30:54.312188 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.312155 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 01:30:54.318250 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.318231 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 01:30:54.318401 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.318377 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-vtffd"] Jul 06 01:30:54.319649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.319608 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 01:30:54.323597 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323577 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-certificates\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.323698 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-bound-sa-token\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.323698 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.323822 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323693 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dfb1195-1355-4f46-8711-fffcb72b9491-ca-trust-extracted\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.323822 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg25t\" (UniqueName: \"kubernetes.io/projected/5fe2ae0a-62b0-4938-a271-bb810c1d533c-kube-api-access-gg25t\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.323822 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323793 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-installation-pull-secrets\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.323992 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qvht\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-kube-api-access-6qvht\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.323992 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323962 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-image-registry-private-configuration\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.323992 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.323989 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.324127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.324018 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-default-certificate\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.324127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.324043 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.324127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.324076 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:30:54.324127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.324103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-stats-auth\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.324312 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.324125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-trusted-ca\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.324312 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.324202 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 01:30:54.324312 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.324279 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs podName:b0a348e4-0203-4974-b406-002dc6a05d58 nodeName:}" failed. No retries permitted until 2026-07-06 01:32:56.324259811 +0000 UTC m=+374.459550308 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs") pod "network-metrics-daemon-cn8h2" (UID: "b0a348e4-0203-4974-b406-002dc6a05d58") : secret "metrics-daemon-secret" not found Jul 06 01:30:54.324754 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.324736 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-hvddn"] Jul 06 01:30:54.424634 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.424605 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-tmp\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.424634 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.424635 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cshd\" (UniqueName: \"kubernetes.io/projected/5cf6d3dc-566f-4abf-9165-59525dcaefa7-kube-api-access-6cshd\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.424825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.424704 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-default-certificate\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.424825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.424736 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cf6d3dc-566f-4abf-9165-59525dcaefa7-trusted-ca\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.424825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.424804 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dfb1195-1355-4f46-8711-fffcb72b9491-ca-trust-extracted\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.425006 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.424840 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg25t\" (UniqueName: \"kubernetes.io/projected/5fe2ae0a-62b0-4938-a271-bb810c1d533c-kube-api-access-gg25t\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.425006 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.424890 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-installation-pull-secrets\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.425104 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425044 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf6d3dc-566f-4abf-9165-59525dcaefa7-config\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.425104 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425069 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf6d3dc-566f-4abf-9165-59525dcaefa7-serving-cert\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.425275 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425114 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-image-registry-private-configuration\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.425275 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425142 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.425275 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425169 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.425275 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.425240 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 01:30:54.425275 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.425243 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 01:30:54.425275 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.425256 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c5cc8998-9s4wf: secret "image-registry-tls" not found Jul 06 01:30:54.425275 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425255 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dfb1195-1355-4f46-8711-fffcb72b9491-ca-trust-extracted\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.425307 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls podName:5dfb1195-1355-4f46-8711-fffcb72b9491 nodeName:}" failed. No retries permitted until 2026-07-06 01:30:54.925287193 +0000 UTC m=+253.060577691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls") pod "image-registry-65c5cc8998-9s4wf" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491") : secret "image-registry-tls" not found Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.425326 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:30:54.92531687 +0000 UTC m=+253.060607354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : secret "router-metrics-certs-default" not found Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425344 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-serving-cert\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425389 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-stats-auth\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425427 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-trusted-ca\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425514 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-certificates\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425539 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425558 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-bound-sa-token\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425582 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425607 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sr68\" (UniqueName: \"kubernetes.io/projected/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-kube-api-access-6sr68\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425635 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-snapshots\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.425838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.425693 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qvht\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-kube-api-access-6qvht\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.426764 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.426087 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:30:54.926071179 +0000 UTC m=+253.061361686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : configmap references non-existent config key: service-ca.crt Jul 06 01:30:54.426764 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.426103 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-certificates\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.426764 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.426598 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-trusted-ca\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.427515 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.427490 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-installation-pull-secrets\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.427607 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.427554 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-image-registry-private-configuration\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.427850 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.427832 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-default-certificate\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.428270 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.428254 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-stats-auth\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.433033 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.432991 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg25t\" (UniqueName: \"kubernetes.io/projected/5fe2ae0a-62b0-4938-a271-bb810c1d533c-kube-api-access-gg25t\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.434047 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.434027 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-bound-sa-token\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.434421 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.434401 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qvht\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-kube-api-access-6qvht\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.526472 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526414 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf6d3dc-566f-4abf-9165-59525dcaefa7-config\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.526472 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526440 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf6d3dc-566f-4abf-9165-59525dcaefa7-serving-cert\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.526472 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526473 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-serving-cert\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.526680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526494 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.526680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526511 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.526680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526530 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sr68\" (UniqueName: \"kubernetes.io/projected/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-kube-api-access-6sr68\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.526680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526578 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-snapshots\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.526680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526607 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-tmp\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.526680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526635 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cshd\" (UniqueName: \"kubernetes.io/projected/5cf6d3dc-566f-4abf-9165-59525dcaefa7-kube-api-access-6cshd\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.526680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.526671 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cf6d3dc-566f-4abf-9165-59525dcaefa7-trusted-ca\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.527138 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.527113 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.527352 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.527208 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5cf6d3dc-566f-4abf-9165-59525dcaefa7-config\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.527352 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.527299 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-snapshots\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.527487 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.527467 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cf6d3dc-566f-4abf-9165-59525dcaefa7-trusted-ca\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.527546 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.527525 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-tmp\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.527639 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.527616 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.528905 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.528883 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-serving-cert\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.528993 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.528945 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5cf6d3dc-566f-4abf-9165-59525dcaefa7-serving-cert\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.535075 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.535057 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cshd\" (UniqueName: \"kubernetes.io/projected/5cf6d3dc-566f-4abf-9165-59525dcaefa7-kube-api-access-6cshd\") pod \"console-operator-68b757fb67-hvddn\" (UID: \"5cf6d3dc-566f-4abf-9165-59525dcaefa7\") " pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.535204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.535184 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sr68\" (UniqueName: \"kubernetes.io/projected/066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc-kube-api-access-6sr68\") pod \"insights-operator-5c847dcfc4-vtffd\" (UID: \"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc\") " pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.616089 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.616068 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" Jul 06 01:30:54.622391 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.622372 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:30:54.745722 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.745694 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-vtffd"] Jul 06 01:30:54.749946 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:30:54.749915 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod066fe0eb_bed2_44c0_b4b4_bc80a71b1dbc.slice/crio-930ee1d49a92768f815478379f738c7812edea07c79ce2e57835b95e4ebc10aa WatchSource:0}: Error finding container 930ee1d49a92768f815478379f738c7812edea07c79ce2e57835b95e4ebc10aa: Status 404 returned error can't find the container with id 930ee1d49a92768f815478379f738c7812edea07c79ce2e57835b95e4ebc10aa Jul 06 01:30:54.758639 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.758617 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-hvddn"] Jul 06 01:30:54.761397 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:30:54.761375 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf6d3dc_566f_4abf_9165_59525dcaefa7.slice/crio-600eecf5ef4beb7b17f9c654a9e26af87a88faa61bcb96640b81ca3156e59824 WatchSource:0}: Error finding container 600eecf5ef4beb7b17f9c654a9e26af87a88faa61bcb96640b81ca3156e59824: Status 404 returned error can't find the container with id 600eecf5ef4beb7b17f9c654a9e26af87a88faa61bcb96640b81ca3156e59824 Jul 06 01:30:54.930661 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.930634 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:54.930818 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.930667 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.930818 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:54.930695 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:54.930818 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.930800 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 01:30:54.930818 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.930802 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 01:30:54.930818 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.930816 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c5cc8998-9s4wf: secret "image-registry-tls" not found Jul 06 01:30:54.931095 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.930841 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:30:55.930823101 +0000 UTC m=+254.066113603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : configmap references non-existent config key: service-ca.crt Jul 06 01:30:54.931095 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.930891 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls podName:5dfb1195-1355-4f46-8711-fffcb72b9491 nodeName:}" failed. No retries permitted until 2026-07-06 01:30:55.930849824 +0000 UTC m=+254.066140310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls") pod "image-registry-65c5cc8998-9s4wf" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491") : secret "image-registry-tls" not found Jul 06 01:30:54.931095 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:54.930920 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:30:55.930909031 +0000 UTC m=+254.066199529 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : secret "router-metrics-certs-default" not found Jul 06 01:30:55.122608 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:55.122577 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" event={"ID":"5cf6d3dc-566f-4abf-9165-59525dcaefa7","Type":"ContainerStarted","Data":"600eecf5ef4beb7b17f9c654a9e26af87a88faa61bcb96640b81ca3156e59824"} Jul 06 01:30:55.123590 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:55.123565 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" event={"ID":"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc","Type":"ContainerStarted","Data":"930ee1d49a92768f815478379f738c7812edea07c79ce2e57835b95e4ebc10aa"} Jul 06 01:30:55.938911 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:55.938870 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:55.939324 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:55.938919 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:55.939324 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:55.938960 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:55.939324 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:55.939020 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 01:30:55.939324 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:55.939043 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c5cc8998-9s4wf: secret "image-registry-tls" not found Jul 06 01:30:55.939324 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:55.939078 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 01:30:55.939324 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:55.939097 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls podName:5dfb1195-1355-4f46-8711-fffcb72b9491 nodeName:}" failed. No retries permitted until 2026-07-06 01:30:57.939077539 +0000 UTC m=+256.074368023 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls") pod "image-registry-65c5cc8998-9s4wf" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491") : secret "image-registry-tls" not found Jul 06 01:30:55.939324 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:55.939115 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:30:57.939104192 +0000 UTC m=+256.074394674 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : configmap references non-existent config key: service-ca.crt Jul 06 01:30:55.939324 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:55.939177 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:30:57.939158533 +0000 UTC m=+256.074449018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : secret "router-metrics-certs-default" not found Jul 06 01:30:57.959508 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:57.959477 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:57.959888 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:57.959544 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:30:57.959888 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:57.959571 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:30:57.959888 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:57.959642 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:31:01.95962457 +0000 UTC m=+260.094915052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : configmap references non-existent config key: service-ca.crt Jul 06 01:30:57.959888 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:57.959666 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 01:30:57.959888 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:57.959694 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 01:30:57.959888 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:57.959709 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c5cc8998-9s4wf: secret "image-registry-tls" not found Jul 06 01:30:57.959888 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:57.959716 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:31:01.959700541 +0000 UTC m=+260.094991041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : secret "router-metrics-certs-default" not found Jul 06 01:30:57.959888 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:57.959743 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls podName:5dfb1195-1355-4f46-8711-fffcb72b9491 nodeName:}" failed. No retries permitted until 2026-07-06 01:31:01.959732848 +0000 UTC m=+260.095023347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls") pod "image-registry-65c5cc8998-9s4wf" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491") : secret "image-registry-tls" not found Jul 06 01:30:58.135214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.135183 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/0.log" Jul 06 01:30:58.135380 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.135228 2574 generic.go:356] "Generic (PLEG): container finished" podID="5cf6d3dc-566f-4abf-9165-59525dcaefa7" containerID="ff013d6628624eec1008c90c39947f8f1a2c0318eba0fd4b3282aabbbaf7b90b" exitCode=255 Jul 06 01:30:58.135380 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.135307 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" event={"ID":"5cf6d3dc-566f-4abf-9165-59525dcaefa7","Type":"ContainerDied","Data":"ff013d6628624eec1008c90c39947f8f1a2c0318eba0fd4b3282aabbbaf7b90b"} Jul 06 01:30:58.135516 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.135497 2574 scope.go:117] "RemoveContainer" containerID="ff013d6628624eec1008c90c39947f8f1a2c0318eba0fd4b3282aabbbaf7b90b" Jul 06 01:30:58.136791 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.136764 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" event={"ID":"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc","Type":"ContainerStarted","Data":"507e030d0982827eafd22572a44aaff80156e1f4308f703463b89b5c9914ebda"} Jul 06 01:30:58.166562 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.166519 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" podStartSLOduration=1.685941506 podStartE2EDuration="4.1665054s" totalImagesPullingTime="2.480563894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:30:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:30:58.166099072 +0000 UTC m=+256.301389580" watchObservedRunningTime="2026-07-06 01:30:58.1665054 +0000 UTC m=+256.301795885" Jul 06 01:30:58.332335 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.332307 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tllrv"] Jul 06 01:30:58.335225 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.335210 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.338358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.338334 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 01:30:58.338483 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.338454 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 01:30:58.338553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.338535 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9xpc9\"" Jul 06 01:30:58.356587 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.356568 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tllrv"] Jul 06 01:30:58.463281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.463255 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f334001b-d1e4-4107-b5c8-1412e573b2aa-crio-socket\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.463281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.463283 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.463435 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.463306 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.463435 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.463365 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f334001b-d1e4-4107-b5c8-1412e573b2aa-data-volume\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.463435 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.463393 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6lhh\" (UniqueName: \"kubernetes.io/projected/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-api-access-n6lhh\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.564262 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.564195 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f334001b-d1e4-4107-b5c8-1412e573b2aa-crio-socket\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.564262 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.564225 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.564262 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.564247 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.564470 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.564285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f334001b-d1e4-4107-b5c8-1412e573b2aa-data-volume\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.564470 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.564310 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f334001b-d1e4-4107-b5c8-1412e573b2aa-crio-socket\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.564470 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:58.564334 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:30:58.564470 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:58.564362 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:30:58.564470 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:58.564405 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:30:59.064385924 +0000 UTC m=+257.199676408 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : configmap "kube-rbac-proxy" not found Jul 06 01:30:58.564470 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.564425 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6lhh\" (UniqueName: \"kubernetes.io/projected/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-api-access-n6lhh\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.564677 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.564524 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f334001b-d1e4-4107-b5c8-1412e573b2aa-data-volume\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:58.564677 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:58.564526 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:30:59.064515214 +0000 UTC m=+257.199805708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : secret "insights-runtime-extractor-tls" not found Jul 06 01:30:58.573274 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:58.573251 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6lhh\" (UniqueName: \"kubernetes.io/projected/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-api-access-n6lhh\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:59.069140 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:59.069108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:59.069140 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:59.069143 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:30:59.069536 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:59.069222 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:30:59.069536 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:59.069241 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:30:59.069536 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:59.069283 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:31:00.069266796 +0000 UTC m=+258.204557277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : configmap "kube-rbac-proxy" not found Jul 06 01:30:59.069536 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:59.069298 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:31:00.069291452 +0000 UTC m=+258.204581934 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : secret "insights-runtime-extractor-tls" not found Jul 06 01:30:59.141009 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:59.140985 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:30:59.141409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:59.141394 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/0.log" Jul 06 01:30:59.141493 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:59.141429 2574 generic.go:356] "Generic (PLEG): container finished" podID="5cf6d3dc-566f-4abf-9165-59525dcaefa7" containerID="fcb08828efd55485df88be20db3a75b03f6d976e9888e64163336bd858bad7f6" exitCode=255 Jul 06 01:30:59.141535 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:59.141518 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" event={"ID":"5cf6d3dc-566f-4abf-9165-59525dcaefa7","Type":"ContainerDied","Data":"fcb08828efd55485df88be20db3a75b03f6d976e9888e64163336bd858bad7f6"} Jul 06 01:30:59.141572 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:59.141563 2574 scope.go:117] "RemoveContainer" containerID="ff013d6628624eec1008c90c39947f8f1a2c0318eba0fd4b3282aabbbaf7b90b" Jul 06 01:30:59.141792 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:30:59.141771 2574 scope.go:117] "RemoveContainer" containerID="fcb08828efd55485df88be20db3a75b03f6d976e9888e64163336bd858bad7f6" Jul 06 01:30:59.141990 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:30:59.141973 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-hvddn_openshift-console-operator(5cf6d3dc-566f-4abf-9165-59525dcaefa7)\"" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" podUID="5cf6d3dc-566f-4abf-9165-59525dcaefa7" Jul 06 01:31:00.077042 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:00.077008 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:00.077042 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:00.077047 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:00.077504 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:00.077130 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:31:00.077504 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:00.077160 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:31:00.077504 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:00.077201 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:31:02.077181155 +0000 UTC m=+260.212471660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : configmap "kube-rbac-proxy" not found Jul 06 01:31:00.077504 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:00.077216 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:31:02.07720983 +0000 UTC m=+260.212500313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : secret "insights-runtime-extractor-tls" not found Jul 06 01:31:00.088820 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:00.088795 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6x7cb_ca72464b-98ac-4773-adec-7a0df6508fa7/dns-node-resolver/0.log" Jul 06 01:31:00.145746 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:00.145724 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:31:00.146072 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:00.146057 2574 scope.go:117] "RemoveContainer" containerID="fcb08828efd55485df88be20db3a75b03f6d976e9888e64163336bd858bad7f6" Jul 06 01:31:00.146204 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:00.146187 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-hvddn_openshift-console-operator(5cf6d3dc-566f-4abf-9165-59525dcaefa7)\"" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" podUID="5cf6d3dc-566f-4abf-9165-59525dcaefa7" Jul 06 01:31:01.091253 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:01.091221 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9hcgl_b1a91258-14a4-45b3-ba6c-41ec9b31b731/node-ca/0.log" Jul 06 01:31:01.891369 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:01.891333 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4q7d7_2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832/serve-healthcheck-canary/0.log" Jul 06 01:31:01.994081 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:01.994046 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:01.994246 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:01.994118 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:31:01.994246 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:01.994136 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:01.994246 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:01.994206 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:31:09.994187816 +0000 UTC m=+268.129478297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : configmap references non-existent config key: service-ca.crt Jul 06 01:31:01.994370 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:01.994249 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 01:31:01.994370 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:01.994257 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 01:31:01.994370 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:01.994271 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c5cc8998-9s4wf: secret "image-registry-tls" not found Jul 06 01:31:01.994370 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:01.994293 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs podName:5fe2ae0a-62b0-4938-a271-bb810c1d533c nodeName:}" failed. No retries permitted until 2026-07-06 01:31:09.9942821 +0000 UTC m=+268.129572582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs") pod "router-default-7995fbff9c-klmcz" (UID: "5fe2ae0a-62b0-4938-a271-bb810c1d533c") : secret "router-metrics-certs-default" not found Jul 06 01:31:01.994370 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:01.994310 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls podName:5dfb1195-1355-4f46-8711-fffcb72b9491 nodeName:}" failed. No retries permitted until 2026-07-06 01:31:09.994298754 +0000 UTC m=+268.129589235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls") pod "image-registry-65c5cc8998-9s4wf" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491") : secret "image-registry-tls" not found Jul 06 01:31:02.094963 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:02.094935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:02.095302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:02.094968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:02.095302 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:02.095061 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:31:02.095302 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:02.095109 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:31:02.095302 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:02.095113 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:31:06.095101158 +0000 UTC m=+264.230391640 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : configmap "kube-rbac-proxy" not found Jul 06 01:31:02.095302 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:02.095148 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:31:06.095139164 +0000 UTC m=+264.230429645 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : secret "insights-runtime-extractor-tls" not found Jul 06 01:31:04.623285 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:04.623255 2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:31:04.623631 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:04.623307 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:31:04.623631 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:04.623586 2574 scope.go:117] "RemoveContainer" containerID="fcb08828efd55485df88be20db3a75b03f6d976e9888e64163336bd858bad7f6" Jul 06 01:31:04.623734 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:04.623715 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-hvddn_openshift-console-operator(5cf6d3dc-566f-4abf-9165-59525dcaefa7)\"" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" podUID="5cf6d3dc-566f-4abf-9165-59525dcaefa7" Jul 06 01:31:06.125448 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:06.125406 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:06.125448 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:06.125460 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:06.125886 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:06.125529 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 01:31:06.125886 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:06.125588 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:31:14.125573669 +0000 UTC m=+272.260864150 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : configmap "kube-rbac-proxy" not found Jul 06 01:31:06.125886 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:06.125607 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 01:31:06.125886 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:06.125644 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls podName:f334001b-d1e4-4107-b5c8-1412e573b2aa nodeName:}" failed. No retries permitted until 2026-07-06 01:31:14.125632648 +0000 UTC m=+272.260923130 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tllrv" (UID: "f334001b-d1e4-4107-b5c8-1412e573b2aa") : secret "insights-runtime-extractor-tls" not found Jul 06 01:31:10.057125 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.057091 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:31:10.057125 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.057127 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:10.057572 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.057172 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:10.057803 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.057785 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5fe2ae0a-62b0-4938-a271-bb810c1d533c-service-ca-bundle\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:10.059492 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.059469 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"image-registry-65c5cc8998-9s4wf\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:31:10.059646 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.059627 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe2ae0a-62b0-4938-a271-bb810c1d533c-metrics-certs\") pod \"router-default-7995fbff9c-klmcz\" (UID: \"5fe2ae0a-62b0-4938-a271-bb810c1d533c\") " pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:10.125405 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.125380 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:10.130274 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.130263 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:31:10.263598 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.263568 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7995fbff9c-klmcz"] Jul 06 01:31:10.266783 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:10.266760 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe2ae0a_62b0_4938_a271_bb810c1d533c.slice/crio-58a7dc5a5231b950529599f5f63cf0d4725c07e4c0c9f285ee2e555bf56aa3ac WatchSource:0}: Error finding container 58a7dc5a5231b950529599f5f63cf0d4725c07e4c0c9f285ee2e555bf56aa3ac: Status 404 returned error can't find the container with id 58a7dc5a5231b950529599f5f63cf0d4725c07e4c0c9f285ee2e555bf56aa3ac Jul 06 01:31:10.278753 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:10.278730 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65c5cc8998-9s4wf"] Jul 06 01:31:10.281966 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:10.281946 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfb1195_1355_4f46_8711_fffcb72b9491.slice/crio-bbae6d3a429e588c192fd603c750fa1dcf71e1eb168612c0a15fd67f8c46572f WatchSource:0}: Error finding container bbae6d3a429e588c192fd603c750fa1dcf71e1eb168612c0a15fd67f8c46572f: Status 404 returned error can't find the container with id bbae6d3a429e588c192fd603c750fa1dcf71e1eb168612c0a15fd67f8c46572f Jul 06 01:31:11.184003 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:11.183968 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" event={"ID":"5dfb1195-1355-4f46-8711-fffcb72b9491","Type":"ContainerStarted","Data":"77c5cf213ec3ad2b0554e1f0da54ee16aafe56aca86ab2098fad0fe5198d6f1b"} Jul 06 01:31:11.184444 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:11.184011 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" event={"ID":"5dfb1195-1355-4f46-8711-fffcb72b9491","Type":"ContainerStarted","Data":"bbae6d3a429e588c192fd603c750fa1dcf71e1eb168612c0a15fd67f8c46572f"} Jul 06 01:31:11.184444 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:11.184051 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:31:11.185536 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:11.185505 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7995fbff9c-klmcz" event={"ID":"5fe2ae0a-62b0-4938-a271-bb810c1d533c","Type":"ContainerStarted","Data":"7c75c4c6c585ab7d8ceeccbbf62556c834e8790faf081e89061ce6b0a4d7632e"} Jul 06 01:31:11.185649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:11.185545 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7995fbff9c-klmcz" event={"ID":"5fe2ae0a-62b0-4938-a271-bb810c1d533c","Type":"ContainerStarted","Data":"58a7dc5a5231b950529599f5f63cf0d4725c07e4c0c9f285ee2e555bf56aa3ac"} Jul 06 01:31:11.202435 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:11.202395 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" podStartSLOduration=17.202381676 podStartE2EDuration="17.202381676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:11.201690796 +0000 UTC m=+269.336981302" watchObservedRunningTime="2026-07-06 01:31:11.202381676 +0000 UTC m=+269.337672160" Jul 06 01:31:11.218952 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:11.218915 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7995fbff9c-klmcz" podStartSLOduration=17.21890493 podStartE2EDuration="17.21890493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:11.217906286 +0000 UTC m=+269.353196792" watchObservedRunningTime="2026-07-06 01:31:11.21890493 +0000 UTC m=+269.354195437" Jul 06 01:31:12.125712 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:12.125677 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:12.129780 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:12.129758 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:12.188324 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:12.188300 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:12.189364 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:12.189347 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7995fbff9c-klmcz" Jul 06 01:31:14.191720 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:14.191687 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:14.192152 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:14.191737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:14.192342 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:14.192322 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f334001b-d1e4-4107-b5c8-1412e573b2aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:14.194271 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:14.194252 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f334001b-d1e4-4107-b5c8-1412e573b2aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tllrv\" (UID: \"f334001b-d1e4-4107-b5c8-1412e573b2aa\") " pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:14.244480 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:14.244443 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tllrv" Jul 06 01:31:14.373939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:14.373910 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tllrv"] Jul 06 01:31:14.377382 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:14.377354 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf334001b_d1e4_4107_b5c8_1412e573b2aa.slice/crio-b870158b7d45a45c9a0c78f73cab7e5e89e9fb86b7cf339e82a72862daa4db28 WatchSource:0}: Error finding container b870158b7d45a45c9a0c78f73cab7e5e89e9fb86b7cf339e82a72862daa4db28: Status 404 returned error can't find the container with id b870158b7d45a45c9a0c78f73cab7e5e89e9fb86b7cf339e82a72862daa4db28 Jul 06 01:31:15.199077 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:15.199038 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tllrv" event={"ID":"f334001b-d1e4-4107-b5c8-1412e573b2aa","Type":"ContainerStarted","Data":"b870158b7d45a45c9a0c78f73cab7e5e89e9fb86b7cf339e82a72862daa4db28"} Jul 06 01:31:17.376750 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:17.376716 2574 scope.go:117] "RemoveContainer" containerID="fcb08828efd55485df88be20db3a75b03f6d976e9888e64163336bd858bad7f6" Jul 06 01:31:18.209668 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:18.209571 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tllrv" event={"ID":"f334001b-d1e4-4107-b5c8-1412e573b2aa","Type":"ContainerStarted","Data":"a79211afd144f288bfd34c62af77fe1df06f5f93ea8bed6baed8f5c10443d9b0"} Jul 06 01:31:18.211365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:18.211345 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:31:18.211486 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:18.211400 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" event={"ID":"5cf6d3dc-566f-4abf-9165-59525dcaefa7","Type":"ContainerStarted","Data":"f7e0d0d0bbc0ea8563474e463ec6c7296d3eabc75404a4b0860149c98379d310"} Jul 06 01:31:18.211660 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:18.211642 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:31:18.226716 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:18.226677 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" podStartSLOduration=21.759374303 podStartE2EDuration="24.22666527s" totalImagesPullingTime="2.467290967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:30:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:18.225589284 +0000 UTC m=+276.360879787" watchObservedRunningTime="2026-07-06 01:31:18.22666527 +0000 UTC m=+276.361955774" Jul 06 01:31:18.863395 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:18.863364 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-hvddn" Jul 06 01:31:19.217598 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:19.217559 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tllrv" event={"ID":"f334001b-d1e4-4107-b5c8-1412e573b2aa","Type":"ContainerStarted","Data":"6223b9cbd5661f66c20fcac3eabce8ed7b90b1c0ec7a6fe413da6d87f5a4f784"} Jul 06 01:31:21.228582 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:21.228549 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tllrv" event={"ID":"f334001b-d1e4-4107-b5c8-1412e573b2aa","Type":"ContainerStarted","Data":"7df2df6dfb678beb82fd8b0777a77159e6daa4b3b4bb1842c2df483b8820e161"} Jul 06 01:31:21.245005 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:21.244962 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tllrv" podStartSLOduration=16.611140473 podStartE2EDuration="23.244946707s" totalImagesPullingTime="6.633806234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 01:30:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:21.24369109 +0000 UTC m=+279.378981595" watchObservedRunningTime="2026-07-06 01:31:21.244946707 +0000 UTC m=+279.380237211" Jul 06 01:31:24.152523 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:24.152492 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65c5cc8998-9s4wf"] Jul 06 01:31:24.862512 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:24.862474 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-dmkmq" podUID="b07fe06c-ab11-4f51-8a99-286d02bad557" Jul 06 01:31:25.241126 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:25.241057 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dmkmq" Jul 06 01:31:28.188551 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.188510 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:31:28.191004 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.190983 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b07fe06c-ab11-4f51-8a99-286d02bad557-metrics-tls\") pod \"dns-default-dmkmq\" (UID: \"b07fe06c-ab11-4f51-8a99-286d02bad557\") " pod="openshift-dns/dns-default-dmkmq" Jul 06 01:31:28.245474 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.245453 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-58ndz\"" Jul 06 01:31:28.252111 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.252079 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dmkmq" Jul 06 01:31:28.391712 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.391687 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dmkmq"] Jul 06 01:31:28.394821 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:28.394790 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07fe06c_ab11_4f51_8a99_286d02bad557.slice/crio-d149130c2df59fda2e5634cde097205e5d200a17b1d4de91ff91f6d1e764415f WatchSource:0}: Error finding container d149130c2df59fda2e5634cde097205e5d200a17b1d4de91ff91f6d1e764415f: Status 404 returned error can't find the container with id d149130c2df59fda2e5634cde097205e5d200a17b1d4de91ff91f6d1e764415f Jul 06 01:31:28.781588 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.781557 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64cdcc98f-dvb9k"] Jul 06 01:31:28.789784 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.789760 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.792218 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792198 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-console-config\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.792358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792223 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-service-ca\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.792358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792242 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-oauth-serving-cert\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.792358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792277 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cgnmd\"" Jul 06 01:31:28.792358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792282 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wm9v\" (UniqueName: \"kubernetes.io/projected/f255014a-02eb-4a0d-babf-37d10ed853bf-kube-api-access-6wm9v\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.792358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792328 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-serving-cert\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.792588 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792367 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-oauth-config\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.792642 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792611 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 01:31:28.792872 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792838 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 01:31:28.792951 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792904 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 01:31:28.792951 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.792919 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 01:31:28.793101 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.793085 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 01:31:28.793192 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.793165 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 01:31:28.793192 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.793168 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 01:31:28.795732 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.795711 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cdcc98f-dvb9k"] Jul 06 01:31:28.892806 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.892779 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wm9v\" (UniqueName: \"kubernetes.io/projected/f255014a-02eb-4a0d-babf-37d10ed853bf-kube-api-access-6wm9v\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.892958 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.892815 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-serving-cert\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.892958 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.892845 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-oauth-config\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.892958 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.892890 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-console-config\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.892958 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.892919 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-service-ca\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.892958 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.892944 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-oauth-serving-cert\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.893688 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.893562 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-console-config\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.893688 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.893598 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-service-ca\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.893912 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.893891 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-oauth-serving-cert\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.895446 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.895426 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-serving-cert\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.895446 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.895436 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-oauth-config\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:28.901653 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:28.901632 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wm9v\" (UniqueName: \"kubernetes.io/projected/f255014a-02eb-4a0d-babf-37d10ed853bf-kube-api-access-6wm9v\") pod \"console-64cdcc98f-dvb9k\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:29.100398 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:29.100367 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:29.248608 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:29.248577 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64cdcc98f-dvb9k"] Jul 06 01:31:29.252260 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:29.252233 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf255014a_02eb_4a0d_babf_37d10ed853bf.slice/crio-ea82c2f3270e4811eac90d44eb9d835cbb0a0f47b3680b9146c3557ce9d5b5ae WatchSource:0}: Error finding container ea82c2f3270e4811eac90d44eb9d835cbb0a0f47b3680b9146c3557ce9d5b5ae: Status 404 returned error can't find the container with id ea82c2f3270e4811eac90d44eb9d835cbb0a0f47b3680b9146c3557ce9d5b5ae Jul 06 01:31:29.255021 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:29.254930 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dmkmq" event={"ID":"b07fe06c-ab11-4f51-8a99-286d02bad557","Type":"ContainerStarted","Data":"d149130c2df59fda2e5634cde097205e5d200a17b1d4de91ff91f6d1e764415f"} Jul 06 01:31:30.262386 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:30.262344 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dmkmq" event={"ID":"b07fe06c-ab11-4f51-8a99-286d02bad557","Type":"ContainerStarted","Data":"3cab84062dc661a0c158fef15bfb65b25b036c37a6d8560f6def982385126e10"} Jul 06 01:31:30.263716 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:30.263686 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cdcc98f-dvb9k" event={"ID":"f255014a-02eb-4a0d-babf-37d10ed853bf","Type":"ContainerStarted","Data":"ea82c2f3270e4811eac90d44eb9d835cbb0a0f47b3680b9146c3557ce9d5b5ae"} Jul 06 01:31:31.269870 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:31.269829 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dmkmq" event={"ID":"b07fe06c-ab11-4f51-8a99-286d02bad557","Type":"ContainerStarted","Data":"283344bd51deca45ff019dfd8950f6523df586874074a7cbec78004aa923c57b"} Jul 06 01:31:31.270396 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:31.269947 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dmkmq" Jul 06 01:31:31.285993 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:31.285955 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dmkmq" podStartSLOduration=251.551225058 podStartE2EDuration="4m13.285941961s" totalImagesPullingTime="1.734716903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:31.284754414 +0000 UTC m=+289.420044918" watchObservedRunningTime="2026-07-06 01:31:31.285941961 +0000 UTC m=+289.421232466" Jul 06 01:31:33.278850 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:33.278814 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cdcc98f-dvb9k" event={"ID":"f255014a-02eb-4a0d-babf-37d10ed853bf","Type":"ContainerStarted","Data":"3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa"} Jul 06 01:31:33.280462 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:33.280439 2574 generic.go:356] "Generic (PLEG): container finished" podID="9f6ef509-306b-4242-b343-63a0bbf92566" containerID="2ab85e2d065de1b3e271252ce638565bab5c042f010d369f722ca332fcf11fec" exitCode=255 Jul 06 01:31:33.280528 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:33.280474 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" event={"ID":"9f6ef509-306b-4242-b343-63a0bbf92566","Type":"ContainerDied","Data":"2ab85e2d065de1b3e271252ce638565bab5c042f010d369f722ca332fcf11fec"} Jul 06 01:31:33.280528 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:33.280504 2574 scope.go:117] "RemoveContainer" containerID="415c9834317ed6d3bdbe878b4145218e92a6b3974083a6d46c0df1e6b1f21a3d" Jul 06 01:31:33.286433 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:33.286414 2574 scope.go:117] "RemoveContainer" containerID="2ab85e2d065de1b3e271252ce638565bab5c042f010d369f722ca332fcf11fec" Jul 06 01:31:33.286580 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:33.286563 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-75bdb96784-f2dxn_open-cluster-management-agent-addon(9f6ef509-306b-4242-b343-63a0bbf92566)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" podUID="9f6ef509-306b-4242-b343-63a0bbf92566" Jul 06 01:31:33.295769 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:33.295732 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64cdcc98f-dvb9k" podStartSLOduration=1.766799179 podStartE2EDuration="5.295719126s" totalImagesPullingTime="3.528919947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:33.294657834 +0000 UTC m=+291.429948338" watchObservedRunningTime="2026-07-06 01:31:33.295719126 +0000 UTC m=+291.431009630" Jul 06 01:31:34.157641 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:34.157604 2574 patch_prober.go:28] interesting pod/image-registry-65c5cc8998-9s4wf container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 01:31:34.157795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:34.157666 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" podUID="5dfb1195-1355-4f46-8711-fffcb72b9491" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 01:31:36.097114 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.097084 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw"] Jul 06 01:31:36.101564 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.101546 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.103555 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.103532 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 01:31:36.104126 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.104105 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-j6qfz\"" Jul 06 01:31:36.104126 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.104111 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 01:31:36.104280 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.104105 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 01:31:36.104280 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.104143 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 01:31:36.104280 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.104125 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 01:31:36.113332 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.113313 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw"] Jul 06 01:31:36.126671 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.126655 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt"] Jul 06 01:31:36.129678 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.129657 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.129947 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.129929 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qjngt"] Jul 06 01:31:36.131559 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.131525 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 01:31:36.131812 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.131796 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 01:31:36.132237 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.132220 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4r6rz\"" Jul 06 01:31:36.132302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.132259 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 01:31:36.133078 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.133062 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.135483 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.135460 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 01:31:36.135579 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.135549 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 01:31:36.135807 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.135786 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tppmq\"" Jul 06 01:31:36.136013 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.135990 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 01:31:36.145641 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.145622 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt"] Jul 06 01:31:36.156541 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156523 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-tls\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.156619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.156619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156571 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.156619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156591 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-root\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.156619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156615 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.156747 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.156747 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156654 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.156747 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156675 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cc3ffe3a-2925-4815-899a-0831d4ab8135-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.156747 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156702 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0580fae-7b59-4076-8312-635b9daf139f-metrics-client-ca\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.156747 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156718 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.156747 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156735 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-wtmp\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.156953 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156754 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-sys\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.156953 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc3ffe3a-2925-4815-899a-0831d4ab8135-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.156953 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.156851 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nffzt\" (UniqueName: \"kubernetes.io/projected/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-api-access-nffzt\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.157243 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.157207 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-textfile\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.157399 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.157383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/646b659c-6b5c-4bf1-859b-3dd93a755697-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.157512 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.157497 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5ph8\" (UniqueName: \"kubernetes.io/projected/646b659c-6b5c-4bf1-859b-3dd93a755697-kube-api-access-p5ph8\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.157627 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.157614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.157728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.157716 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrr4\" (UniqueName: \"kubernetes.io/projected/d0580fae-7b59-4076-8312-635b9daf139f-kube-api-access-4xrr4\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.258756 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.258726 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-root\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.258756 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.258759 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.258965 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.258783 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.258965 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.258811 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.258965 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.258831 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-root\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.258965 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:36.258932 2574 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 01:31:36.259112 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:36.258982 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-tls podName:cc3ffe3a-2925-4815-899a-0831d4ab8135 nodeName:}" failed. No retries permitted until 2026-07-06 01:31:36.758967193 +0000 UTC m=+294.894257676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-7xpnt" (UID: "cc3ffe3a-2925-4815-899a-0831d4ab8135") : secret "kube-state-metrics-tls" not found Jul 06 01:31:36.259112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.258934 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cc3ffe3a-2925-4815-899a-0831d4ab8135-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.259112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259033 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0580fae-7b59-4076-8312-635b9daf139f-metrics-client-ca\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259059 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.259112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259087 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-wtmp\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259111 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-sys\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259150 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc3ffe3a-2925-4815-899a-0831d4ab8135-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nffzt\" (UniqueName: \"kubernetes.io/projected/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-api-access-nffzt\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259215 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-textfile\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259237 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/646b659c-6b5c-4bf1-859b-3dd93a755697-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259261 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5ph8\" (UniqueName: \"kubernetes.io/projected/646b659c-6b5c-4bf1-859b-3dd93a755697-kube-api-access-p5ph8\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259274 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cc3ffe3a-2925-4815-899a-0831d4ab8135-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259302 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:36.259328 2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 06 01:31:36.259362 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrr4\" (UniqueName: \"kubernetes.io/projected/d0580fae-7b59-4076-8312-635b9daf139f-kube-api-access-4xrr4\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:36.259376 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-tls podName:646b659c-6b5c-4bf1-859b-3dd93a755697 nodeName:}" failed. No retries permitted until 2026-07-06 01:31:36.759360913 +0000 UTC m=+294.894651445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-fmwbw" (UID: "646b659c-6b5c-4bf1-859b-3dd93a755697") : secret "openshift-state-metrics-tls" not found Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-tls\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259262 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-wtmp\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259464 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259495 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259519 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-textfile\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259524 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.259579 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d0580fae-7b59-4076-8312-635b9daf139f-sys\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:36.259733 2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 01:31:36.259778 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:36.259783 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-tls podName:d0580fae-7b59-4076-8312-635b9daf139f nodeName:}" failed. No retries permitted until 2026-07-06 01:31:36.759769628 +0000 UTC m=+294.895060119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-tls") pod "node-exporter-qjngt" (UID: "d0580fae-7b59-4076-8312-635b9daf139f") : secret "node-exporter-tls" not found Jul 06 01:31:36.260295 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.260013 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0580fae-7b59-4076-8312-635b9daf139f-metrics-client-ca\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.260295 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.260121 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/646b659c-6b5c-4bf1-859b-3dd93a755697-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.260295 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.260257 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc3ffe3a-2925-4815-899a-0831d4ab8135-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.261817 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.261797 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.261976 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.261960 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.262034 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.261978 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.267443 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.267419 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrr4\" (UniqueName: \"kubernetes.io/projected/d0580fae-7b59-4076-8312-635b9daf139f-kube-api-access-4xrr4\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.268042 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.268020 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nffzt\" (UniqueName: \"kubernetes.io/projected/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-api-access-nffzt\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.268270 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.268251 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5ph8\" (UniqueName: \"kubernetes.io/projected/646b659c-6b5c-4bf1-859b-3dd93a755697-kube-api-access-p5ph8\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.763593 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.763556 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-tls\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.763593 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.763601 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:36.763812 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.763622 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.766164 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.766129 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/646b659c-6b5c-4bf1-859b-3dd93a755697-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-fmwbw\" (UID: \"646b659c-6b5c-4bf1-859b-3dd93a755697\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:36.766164 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.766148 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d0580fae-7b59-4076-8312-635b9daf139f-node-exporter-tls\") pod \"node-exporter-qjngt\" (UID: \"d0580fae-7b59-4076-8312-635b9daf139f\") " pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:36.766164 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:36.766160 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cc3ffe3a-2925-4815-899a-0831d4ab8135-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-7xpnt\" (UID: \"cc3ffe3a-2925-4815-899a-0831d4ab8135\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:37.026177 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.026099 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" Jul 06 01:31:37.039338 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.039308 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" Jul 06 01:31:37.043198 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.043179 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qjngt" Jul 06 01:31:37.055281 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:37.055243 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0580fae_7b59_4076_8312_635b9daf139f.slice/crio-797289e091f992527717baddec34c15991407930336ba3d8c3cd933cb1bd89e3 WatchSource:0}: Error finding container 797289e091f992527717baddec34c15991407930336ba3d8c3cd933cb1bd89e3: Status 404 returned error can't find the container with id 797289e091f992527717baddec34c15991407930336ba3d8c3cd933cb1bd89e3 Jul 06 01:31:37.167421 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.167369 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw"] Jul 06 01:31:37.170177 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:37.170153 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646b659c_6b5c_4bf1_859b_3dd93a755697.slice/crio-c132460368430a09f9350f756f207b5b62436bfe3ca0da2cb80d8869f4a72c76 WatchSource:0}: Error finding container c132460368430a09f9350f756f207b5b62436bfe3ca0da2cb80d8869f4a72c76: Status 404 returned error can't find the container with id c132460368430a09f9350f756f207b5b62436bfe3ca0da2cb80d8869f4a72c76 Jul 06 01:31:37.204030 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.204005 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt"] Jul 06 01:31:37.207138 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:37.207114 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc3ffe3a_2925_4815_899a_0831d4ab8135.slice/crio-bee273758c13a41e69ccfee9ffa324b8af78a254961ff39dff7ad6e82f18258e WatchSource:0}: Error finding container bee273758c13a41e69ccfee9ffa324b8af78a254961ff39dff7ad6e82f18258e: Status 404 returned error can't find the container with id bee273758c13a41e69ccfee9ffa324b8af78a254961ff39dff7ad6e82f18258e Jul 06 01:31:37.296776 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.296744 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" event={"ID":"cc3ffe3a-2925-4815-899a-0831d4ab8135","Type":"ContainerStarted","Data":"bee273758c13a41e69ccfee9ffa324b8af78a254961ff39dff7ad6e82f18258e"} Jul 06 01:31:37.298724 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.298699 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" event={"ID":"646b659c-6b5c-4bf1-859b-3dd93a755697","Type":"ContainerStarted","Data":"37d32c59cae521f8080dca2c506f368e9a705519fe004fd1554aa32e2f2cd7e5"} Jul 06 01:31:37.298838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.298748 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" event={"ID":"646b659c-6b5c-4bf1-859b-3dd93a755697","Type":"ContainerStarted","Data":"8cb6ba47c3f7d73e4bbcbfa0b5dbeccc61adf7eafca2caea034e2ba0fd3a9c75"} Jul 06 01:31:37.298838 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.298767 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" event={"ID":"646b659c-6b5c-4bf1-859b-3dd93a755697","Type":"ContainerStarted","Data":"c132460368430a09f9350f756f207b5b62436bfe3ca0da2cb80d8869f4a72c76"} Jul 06 01:31:37.299917 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.299888 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjngt" event={"ID":"d0580fae-7b59-4076-8312-635b9daf139f","Type":"ContainerStarted","Data":"797289e091f992527717baddec34c15991407930336ba3d8c3cd933cb1bd89e3"} Jul 06 01:31:37.346607 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.346576 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-588565fd56-dv2kk"] Jul 06 01:31:37.349637 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.349623 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.356451 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.356429 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 01:31:37.360278 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.360260 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588565fd56-dv2kk"] Jul 06 01:31:37.367439 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.367417 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-config\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.367526 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.367461 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-serving-cert\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.367585 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.367570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-oauth-serving-cert\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.367630 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.367617 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-service-ca\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.367679 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.367664 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxwvq\" (UniqueName: \"kubernetes.io/projected/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-kube-api-access-hxwvq\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.367717 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.367708 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-trusted-ca-bundle\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.367755 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.367739 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-oauth-config\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.377462 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:37.377437 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cn8h2" podUID="b0a348e4-0203-4974-b406-002dc6a05d58" Jul 06 01:31:37.468682 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.468658 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-serving-cert\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.468802 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.468707 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-oauth-serving-cert\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.468802 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.468736 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-service-ca\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.468802 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.468762 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxwvq\" (UniqueName: \"kubernetes.io/projected/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-kube-api-access-hxwvq\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.468802 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.468794 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-trusted-ca-bundle\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.469026 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.468823 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-oauth-config\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.469026 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.468850 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-config\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.469478 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.469438 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-service-ca\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.469587 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.469508 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-oauth-serving-cert\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.469660 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.469639 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-config\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.469843 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.469822 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-trusted-ca-bundle\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.471301 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.471276 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-oauth-config\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.471446 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.471429 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-serving-cert\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.475962 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.475943 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxwvq\" (UniqueName: \"kubernetes.io/projected/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-kube-api-access-hxwvq\") pod \"console-588565fd56-dv2kk\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.659701 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.659674 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:37.803941 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:37.803909 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588565fd56-dv2kk"] Jul 06 01:31:38.023348 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:38.023274 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5efde4bb_b7fb_4553_a3bc_2977126ea8e0.slice/crio-937bf44decbf710375b4d4d7b7d2d6fed5ee9cb698c835d62400b1084d24b685 WatchSource:0}: Error finding container 937bf44decbf710375b4d4d7b7d2d6fed5ee9cb698c835d62400b1084d24b685: Status 404 returned error can't find the container with id 937bf44decbf710375b4d4d7b7d2d6fed5ee9cb698c835d62400b1084d24b685 Jul 06 01:31:38.314327 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:38.314230 2574 generic.go:356] "Generic (PLEG): container finished" podID="d0580fae-7b59-4076-8312-635b9daf139f" containerID="374be82d49533ef6f50a6f0bb6b09471a77717b5f29c5c3623cd6edda7adc2f4" exitCode=0 Jul 06 01:31:38.314846 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:38.314725 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjngt" event={"ID":"d0580fae-7b59-4076-8312-635b9daf139f","Type":"ContainerDied","Data":"374be82d49533ef6f50a6f0bb6b09471a77717b5f29c5c3623cd6edda7adc2f4"} Jul 06 01:31:38.317649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:38.317267 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588565fd56-dv2kk" event={"ID":"5efde4bb-b7fb-4553-a3bc-2977126ea8e0","Type":"ContainerStarted","Data":"6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d"} Jul 06 01:31:38.317790 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:38.317685 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588565fd56-dv2kk" event={"ID":"5efde4bb-b7fb-4553-a3bc-2977126ea8e0","Type":"ContainerStarted","Data":"937bf44decbf710375b4d4d7b7d2d6fed5ee9cb698c835d62400b1084d24b685"} Jul 06 01:31:38.357097 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:38.357047 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-588565fd56-dv2kk" podStartSLOduration=1.357030565 podStartE2EDuration="1.357030565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:38.356421241 +0000 UTC m=+296.491711745" watchObservedRunningTime="2026-07-06 01:31:38.357030565 +0000 UTC m=+296.492321069" Jul 06 01:31:38.550101 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:38.550066 2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" Jul 06 01:31:38.557723 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:38.557699 2574 scope.go:117] "RemoveContainer" containerID="2ab85e2d065de1b3e271252ce638565bab5c042f010d369f722ca332fcf11fec" Jul 06 01:31:38.557921 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:31:38.557898 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-75bdb96784-f2dxn_open-cluster-management-agent-addon(9f6ef509-306b-4242-b343-63a0bbf92566)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" podUID="9f6ef509-306b-4242-b343-63a0bbf92566" Jul 06 01:31:39.100737 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.100658 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:39.100928 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.100749 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:39.106619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.106595 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:39.193020 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.192956 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-fcfd97664-99kxx"] Jul 06 01:31:39.196250 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.196236 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.200733 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.200707 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4tpp8\"" Jul 06 01:31:39.200901 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.200756 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 01:31:39.201162 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.201142 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 01:31:39.201269 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.201219 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 01:31:39.201321 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.201277 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 01:31:39.201449 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.201437 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b0faeq28jo7ac\"" Jul 06 01:31:39.201743 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.201730 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 01:31:39.219004 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.218879 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-fcfd97664-99kxx"] Jul 06 01:31:39.287796 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.287762 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.287796 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.287799 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e885ed9c-32c5-4047-9217-9db6832fc046-metrics-client-ca\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.287995 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.287827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.287995 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.287884 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf59f\" (UniqueName: \"kubernetes.io/projected/e885ed9c-32c5-4047-9217-9db6832fc046-kube-api-access-xf59f\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.287995 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.287936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-tls\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.287995 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.287983 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.288163 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.288024 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-grpc-tls\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.288163 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.288085 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.322777 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.322743 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" event={"ID":"646b659c-6b5c-4bf1-859b-3dd93a755697","Type":"ContainerStarted","Data":"50e35cb7c2cad87ad900c0bcacae4ab87a5fa56799abec23a6f8a604f65c706f"} Jul 06 01:31:39.324994 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.324970 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjngt" event={"ID":"d0580fae-7b59-4076-8312-635b9daf139f","Type":"ContainerStarted","Data":"bbeedcaf6f01de16655d5c29e993fddb52cb00dda088f0aca97e9fcb61baa1de"} Jul 06 01:31:39.325121 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.325017 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjngt" event={"ID":"d0580fae-7b59-4076-8312-635b9daf139f","Type":"ContainerStarted","Data":"a0e52619280864bd6e6b13d4d42602cb273373505c0ef5086920f79e0bf18ac2"} Jul 06 01:31:39.327178 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.327153 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" event={"ID":"cc3ffe3a-2925-4815-899a-0831d4ab8135","Type":"ContainerStarted","Data":"0a3f43b56245a1e24b1a0db07a4d02042aa273f764f3d6c171e3914bc585db47"} Jul 06 01:31:39.327288 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.327201 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" event={"ID":"cc3ffe3a-2925-4815-899a-0831d4ab8135","Type":"ContainerStarted","Data":"709f3e4f7da171e9f81da8b3c338a9c21537f6facd51bb6e7f4e172c88c7126e"} Jul 06 01:31:39.327288 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.327228 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" event={"ID":"cc3ffe3a-2925-4815-899a-0831d4ab8135","Type":"ContainerStarted","Data":"7137e14109630644c4f95ad25d391f295bbda815d41cbbf78c40b88c7be6092d"} Jul 06 01:31:39.331132 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.331112 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:31:39.343082 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.342643 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-fmwbw" podStartSLOduration=1.751126831 podStartE2EDuration="3.342625632s" totalImagesPullingTime="1.591498801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:39.341603324 +0000 UTC m=+297.476893830" watchObservedRunningTime="2026-07-06 01:31:39.342625632 +0000 UTC m=+297.477916138" Jul 06 01:31:39.380573 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.380520 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-7xpnt" podStartSLOduration=1.7184289179999999 podStartE2EDuration="3.38050385s" totalImagesPullingTime="1.662074932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:39.379310061 +0000 UTC m=+297.514600566" watchObservedRunningTime="2026-07-06 01:31:39.38050385 +0000 UTC m=+297.515794354" Jul 06 01:31:39.389062 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.388726 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.389062 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.388767 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e885ed9c-32c5-4047-9217-9db6832fc046-metrics-client-ca\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.389062 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.388889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.389062 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.389057 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf59f\" (UniqueName: \"kubernetes.io/projected/e885ed9c-32c5-4047-9217-9db6832fc046-kube-api-access-xf59f\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.389344 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.389093 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-tls\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.389344 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.389208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.389474 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.389362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-grpc-tls\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.389474 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.389419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.390411 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.389695 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e885ed9c-32c5-4047-9217-9db6832fc046-metrics-client-ca\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.393231 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.393191 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.393918 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.393845 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.394044 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.394019 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.394122 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.394041 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-grpc-tls\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.394347 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.394330 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.394565 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.394543 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e885ed9c-32c5-4047-9217-9db6832fc046-secret-thanos-querier-tls\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.397558 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.397539 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf59f\" (UniqueName: \"kubernetes.io/projected/e885ed9c-32c5-4047-9217-9db6832fc046-kube-api-access-xf59f\") pod \"thanos-querier-fcfd97664-99kxx\" (UID: \"e885ed9c-32c5-4047-9217-9db6832fc046\") " pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.407395 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.401845 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qjngt" podStartSLOduration=2.3106299679999998 podStartE2EDuration="3.401821484s" totalImagesPullingTime="1.011620502s" totalInitContainerRuntime="79.571014ms" isStatefulPod=true podCreationTimestamp="2026-07-06 01:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:39.400159907 +0000 UTC m=+297.535450412" watchObservedRunningTime="2026-07-06 01:31:39.401821484 +0000 UTC m=+297.537111992" Jul 06 01:31:39.517138 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.517067 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:39.657788 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:39.657754 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-fcfd97664-99kxx"] Jul 06 01:31:39.661426 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:39.661402 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode885ed9c_32c5_4047_9217_9db6832fc046.slice/crio-7c5ce02a7dddc99a6415dfb37e606b2973d5614461bc49471ab49ac9f495006d WatchSource:0}: Error finding container 7c5ce02a7dddc99a6415dfb37e606b2973d5614461bc49471ab49ac9f495006d: Status 404 returned error can't find the container with id 7c5ce02a7dddc99a6415dfb37e606b2973d5614461bc49471ab49ac9f495006d Jul 06 01:31:40.332077 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:40.332042 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" event={"ID":"e885ed9c-32c5-4047-9217-9db6832fc046","Type":"ContainerStarted","Data":"7c5ce02a7dddc99a6415dfb37e606b2973d5614461bc49471ab49ac9f495006d"} Jul 06 01:31:41.276617 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.276583 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dmkmq" Jul 06 01:31:41.369779 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.369746 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-f8479dc67-h8wt8"] Jul 06 01:31:41.374379 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.374340 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.376792 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.376759 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 01:31:41.376931 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.376793 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 01:31:41.376931 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.376847 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8qflc\"" Jul 06 01:31:41.376931 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.376884 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 01:31:41.377185 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.377156 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 01:31:41.377266 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.377164 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 01:31:41.381386 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.381367 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 01:31:41.388032 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.388014 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f8479dc67-h8wt8"] Jul 06 01:31:41.409379 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.409337 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-telemeter-client-tls\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.409502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.409381 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-metrics-client-ca\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.409502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.409454 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-serving-certs-ca-bundle\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.409724 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.409604 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.409724 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.409679 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggctg\" (UniqueName: \"kubernetes.io/projected/591d82c6-fa27-40a3-8757-72e37e9e13f4-kube-api-access-ggctg\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.409850 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.409830 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-federate-client-tls\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.409931 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.409912 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-secret-telemeter-client\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.409975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.409946 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.510594 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.510558 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-federate-client-tls\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.510767 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.510617 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-secret-telemeter-client\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.510767 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.510649 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.510767 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.510729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-telemeter-client-tls\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.510767 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.510754 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-metrics-client-ca\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.511027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.510785 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-serving-certs-ca-bundle\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.511027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.510891 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.511027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.510931 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggctg\" (UniqueName: \"kubernetes.io/projected/591d82c6-fa27-40a3-8757-72e37e9e13f4-kube-api-access-ggctg\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.511782 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.511756 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-metrics-client-ca\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.511922 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.511896 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-serving-certs-ca-bundle\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.511982 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.511940 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/591d82c6-fa27-40a3-8757-72e37e9e13f4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.513751 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.513724 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-secret-telemeter-client\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.513845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.513768 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.513941 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.513901 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-telemeter-client-tls\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.513941 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.513924 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/591d82c6-fa27-40a3-8757-72e37e9e13f4-federate-client-tls\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.518353 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.518314 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggctg\" (UniqueName: \"kubernetes.io/projected/591d82c6-fa27-40a3-8757-72e37e9e13f4-kube-api-access-ggctg\") pod \"telemeter-client-f8479dc67-h8wt8\" (UID: \"591d82c6-fa27-40a3-8757-72e37e9e13f4\") " pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.685930 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.685668 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" Jul 06 01:31:41.833670 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:41.833645 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f8479dc67-h8wt8"] Jul 06 01:31:41.839990 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:31:41.839963 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591d82c6_fa27_40a3_8757_72e37e9e13f4.slice/crio-bded74311e647a6c8e356cbb91fa8ac188bc3f70dcf42ba8e9e156a937d8d5c1 WatchSource:0}: Error finding container bded74311e647a6c8e356cbb91fa8ac188bc3f70dcf42ba8e9e156a937d8d5c1: Status 404 returned error can't find the container with id bded74311e647a6c8e356cbb91fa8ac188bc3f70dcf42ba8e9e156a937d8d5c1 Jul 06 01:31:42.251901 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.251841 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:31:42.252368 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.252346 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:31:42.258809 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.258789 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:31:42.259210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.259191 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:31:42.263416 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.263392 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 01:31:42.343007 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.342976 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" event={"ID":"e885ed9c-32c5-4047-9217-9db6832fc046","Type":"ContainerStarted","Data":"5e8c5ab291aded92e8cb769d34a2eabd5f3ea824f83ab85876ad5cc7962fb8a4"} Jul 06 01:31:42.346406 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.343089 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" event={"ID":"e885ed9c-32c5-4047-9217-9db6832fc046","Type":"ContainerStarted","Data":"a1fd3fba5bbebf592cfed3a0cd3291f418d370446deb815ca379eaab85ec5f2e"} Jul 06 01:31:42.346406 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.343132 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" event={"ID":"e885ed9c-32c5-4047-9217-9db6832fc046","Type":"ContainerStarted","Data":"f5d69b4508b28ad7e62668b49e17f37ecb92b736af50a06de4f2bdc4f4471d5a"} Jul 06 01:31:42.346406 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:42.344105 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" event={"ID":"591d82c6-fa27-40a3-8757-72e37e9e13f4","Type":"ContainerStarted","Data":"bded74311e647a6c8e356cbb91fa8ac188bc3f70dcf42ba8e9e156a937d8d5c1"} Jul 06 01:31:43.351445 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:43.351408 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" event={"ID":"e885ed9c-32c5-4047-9217-9db6832fc046","Type":"ContainerStarted","Data":"95b17020558c15adf09c57cd0fe775413fd6da4f3b0b601c2901e715bd7a434f"} Jul 06 01:31:43.351830 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:43.351485 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" event={"ID":"e885ed9c-32c5-4047-9217-9db6832fc046","Type":"ContainerStarted","Data":"7144bf74a61e334eed275dbda37e9554d8b714d84ad251f52009b2cd4cdffd02"} Jul 06 01:31:43.351830 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:43.351524 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" event={"ID":"e885ed9c-32c5-4047-9217-9db6832fc046","Type":"ContainerStarted","Data":"32940c8ef6e9b2968122d7023e96be273be60b30c1b45103e8b93f4d5d08bc45"} Jul 06 01:31:43.351830 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:43.351568 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:43.373276 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:43.373207 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" podStartSLOduration=1.434780527 podStartE2EDuration="4.373191239s" totalImagesPullingTime="2.938410712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:43.371515842 +0000 UTC m=+301.506806347" watchObservedRunningTime="2026-07-06 01:31:43.373191239 +0000 UTC m=+301.508481743" Jul 06 01:31:43.576776 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:43.576744 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-588565fd56-dv2kk"] Jul 06 01:31:43.970852 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:43.970834 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:31:44.157991 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:44.157927 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:31:44.358571 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:44.358543 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" event={"ID":"591d82c6-fa27-40a3-8757-72e37e9e13f4","Type":"ContainerStarted","Data":"b779a09cdd863365ae37ac78a7eb44d79aafe93f5e95e70f92a77ad7ed87fbd2"} Jul 06 01:31:45.364764 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:45.364733 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" event={"ID":"591d82c6-fa27-40a3-8757-72e37e9e13f4","Type":"ContainerStarted","Data":"ffc04eea49fa1be36472f98757c42fb2d1b9fac8c35bc76a02ee13e6c77cdc33"} Jul 06 01:31:45.365112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:45.364783 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" event={"ID":"591d82c6-fa27-40a3-8757-72e37e9e13f4","Type":"ContainerStarted","Data":"6ba6ab019f38db89ac9f7ab79d12b22035664af95f0a54ea023e1601c2e3d1c6"} Jul 06 01:31:45.385219 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:45.385180 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-f8479dc67-h8wt8" podStartSLOduration=1.290291935 podStartE2EDuration="4.385165995s" totalImagesPullingTime="3.09487406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:31:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:31:45.384218468 +0000 UTC m=+303.519508971" watchObservedRunningTime="2026-07-06 01:31:45.385165995 +0000 UTC m=+303.520456499" Jul 06 01:31:47.660514 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:47.660486 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:31:49.172083 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.172049 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" podUID="5dfb1195-1355-4f46-8711-fffcb72b9491" containerName="registry" containerID="cri-o://77c5cf213ec3ad2b0554e1f0da54ee16aafe56aca86ab2098fad0fe5198d6f1b" gracePeriod=30 Jul 06 01:31:49.364821 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.364799 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-fcfd97664-99kxx" Jul 06 01:31:49.382956 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.382922 2574 generic.go:356] "Generic (PLEG): container finished" podID="5dfb1195-1355-4f46-8711-fffcb72b9491" containerID="77c5cf213ec3ad2b0554e1f0da54ee16aafe56aca86ab2098fad0fe5198d6f1b" exitCode=0 Jul 06 01:31:49.383089 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.382962 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" event={"ID":"5dfb1195-1355-4f46-8711-fffcb72b9491","Type":"ContainerDied","Data":"77c5cf213ec3ad2b0554e1f0da54ee16aafe56aca86ab2098fad0fe5198d6f1b"} Jul 06 01:31:49.417760 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.417738 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:31:49.483156 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483087 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-certificates\") pod \"5dfb1195-1355-4f46-8711-fffcb72b9491\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " Jul 06 01:31:49.483302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483157 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-installation-pull-secrets\") pod \"5dfb1195-1355-4f46-8711-fffcb72b9491\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " Jul 06 01:31:49.483302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483183 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qvht\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-kube-api-access-6qvht\") pod \"5dfb1195-1355-4f46-8711-fffcb72b9491\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " Jul 06 01:31:49.483302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483228 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-bound-sa-token\") pod \"5dfb1195-1355-4f46-8711-fffcb72b9491\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " Jul 06 01:31:49.483302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483254 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-image-registry-private-configuration\") pod \"5dfb1195-1355-4f46-8711-fffcb72b9491\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " Jul 06 01:31:49.483302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483283 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") pod \"5dfb1195-1355-4f46-8711-fffcb72b9491\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " Jul 06 01:31:49.483562 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483332 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-trusted-ca\") pod \"5dfb1195-1355-4f46-8711-fffcb72b9491\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " Jul 06 01:31:49.483562 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483356 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dfb1195-1355-4f46-8711-fffcb72b9491-ca-trust-extracted\") pod \"5dfb1195-1355-4f46-8711-fffcb72b9491\" (UID: \"5dfb1195-1355-4f46-8711-fffcb72b9491\") " Jul 06 01:31:49.484654 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483536 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5dfb1195-1355-4f46-8711-fffcb72b9491" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:31:49.484654 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483772 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5dfb1195-1355-4f46-8711-fffcb72b9491" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:31:49.484654 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.483780 2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-certificates\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:31:49.485900 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.485837 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5dfb1195-1355-4f46-8711-fffcb72b9491" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:31:49.486027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.485934 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-kube-api-access-6qvht" (OuterVolumeSpecName: "kube-api-access-6qvht") pod "5dfb1195-1355-4f46-8711-fffcb72b9491" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491"). InnerVolumeSpecName "kube-api-access-6qvht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:31:49.486027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.485955 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5dfb1195-1355-4f46-8711-fffcb72b9491" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:31:49.486234 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.486208 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5dfb1195-1355-4f46-8711-fffcb72b9491" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:31:49.486620 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.486598 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5dfb1195-1355-4f46-8711-fffcb72b9491" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:31:49.492798 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.492779 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dfb1195-1355-4f46-8711-fffcb72b9491-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5dfb1195-1355-4f46-8711-fffcb72b9491" (UID: "5dfb1195-1355-4f46-8711-fffcb72b9491"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:31:49.584713 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.584680 2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-installation-pull-secrets\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:31:49.584713 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.584708 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qvht\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-kube-api-access-6qvht\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:31:49.584935 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.584721 2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-bound-sa-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:31:49.584935 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.584735 2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5dfb1195-1355-4f46-8711-fffcb72b9491-image-registry-private-configuration\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:31:49.584935 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.584749 2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5dfb1195-1355-4f46-8711-fffcb72b9491-registry-tls\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:31:49.584935 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.584760 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dfb1195-1355-4f46-8711-fffcb72b9491-trusted-ca\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:31:49.584935 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:49.584769 2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5dfb1195-1355-4f46-8711-fffcb72b9491-ca-trust-extracted\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:31:50.387693 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:50.387652 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" event={"ID":"5dfb1195-1355-4f46-8711-fffcb72b9491","Type":"ContainerDied","Data":"bbae6d3a429e588c192fd603c750fa1dcf71e1eb168612c0a15fd67f8c46572f"} Jul 06 01:31:50.387693 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:50.387671 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c5cc8998-9s4wf" Jul 06 01:31:50.388243 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:50.387705 2574 scope.go:117] "RemoveContainer" containerID="77c5cf213ec3ad2b0554e1f0da54ee16aafe56aca86ab2098fad0fe5198d6f1b" Jul 06 01:31:50.402534 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:50.402510 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65c5cc8998-9s4wf"] Jul 06 01:31:50.407214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:50.407192 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65c5cc8998-9s4wf"] Jul 06 01:31:52.377287 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:52.377257 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:31:52.380537 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:52.380513 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dfb1195-1355-4f46-8711-fffcb72b9491" path="/var/lib/kubelet/pods/5dfb1195-1355-4f46-8711-fffcb72b9491/volumes" Jul 06 01:31:52.383467 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:52.383454 2574 scope.go:117] "RemoveContainer" containerID="2ab85e2d065de1b3e271252ce638565bab5c042f010d369f722ca332fcf11fec" Jul 06 01:31:53.401243 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:53.401208 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-75bdb96784-f2dxn" event={"ID":"9f6ef509-306b-4242-b343-63a0bbf92566","Type":"ContainerStarted","Data":"4b02662022b1f59b619825637037512d4e3f6c41fb7c41cd7a97322486fd6079"} Jul 06 01:31:56.612067 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:31:56.612037 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64cdcc98f-dvb9k"] Jul 06 01:32:08.603453 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.603396 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-588565fd56-dv2kk" podUID="5efde4bb-b7fb-4553-a3bc-2977126ea8e0" containerName="console" containerID="cri-o://6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d" gracePeriod=15 Jul 06 01:32:08.865428 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.865408 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:32:08.932784 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.932757 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-config\") pod \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " Jul 06 01:32:08.932784 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.932785 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-oauth-config\") pod \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " Jul 06 01:32:08.933003 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.932839 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-oauth-serving-cert\") pod \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " Jul 06 01:32:08.933003 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.932960 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-trusted-ca-bundle\") pod \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " Jul 06 01:32:08.933105 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933021 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxwvq\" (UniqueName: \"kubernetes.io/projected/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-kube-api-access-hxwvq\") pod \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " Jul 06 01:32:08.933105 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933088 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-serving-cert\") pod \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " Jul 06 01:32:08.933204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933115 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-service-ca\") pod \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\" (UID: \"5efde4bb-b7fb-4553-a3bc-2977126ea8e0\") " Jul 06 01:32:08.933204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933147 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5efde4bb-b7fb-4553-a3bc-2977126ea8e0" (UID: "5efde4bb-b7fb-4553-a3bc-2977126ea8e0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:08.933204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933163 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-config" (OuterVolumeSpecName: "console-config") pod "5efde4bb-b7fb-4553-a3bc-2977126ea8e0" (UID: "5efde4bb-b7fb-4553-a3bc-2977126ea8e0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:08.933337 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933319 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5efde4bb-b7fb-4553-a3bc-2977126ea8e0" (UID: "5efde4bb-b7fb-4553-a3bc-2977126ea8e0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:08.933496 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933474 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-config\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:08.933594 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933499 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-oauth-serving-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:08.933594 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933514 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-trusted-ca-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:08.933594 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.933543 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-service-ca" (OuterVolumeSpecName: "service-ca") pod "5efde4bb-b7fb-4553-a3bc-2977126ea8e0" (UID: "5efde4bb-b7fb-4553-a3bc-2977126ea8e0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:08.935145 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.935116 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5efde4bb-b7fb-4553-a3bc-2977126ea8e0" (UID: "5efde4bb-b7fb-4553-a3bc-2977126ea8e0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:32:08.935760 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.935737 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5efde4bb-b7fb-4553-a3bc-2977126ea8e0" (UID: "5efde4bb-b7fb-4553-a3bc-2977126ea8e0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:32:08.935760 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:08.935740 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-kube-api-access-hxwvq" (OuterVolumeSpecName: "kube-api-access-hxwvq") pod "5efde4bb-b7fb-4553-a3bc-2977126ea8e0" (UID: "5efde4bb-b7fb-4553-a3bc-2977126ea8e0"). InnerVolumeSpecName "kube-api-access-hxwvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:32:09.034779 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.034756 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-serving-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:09.034779 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.034779 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-service-ca\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:09.034950 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.034788 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-console-oauth-config\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:09.034950 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.034796 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxwvq\" (UniqueName: \"kubernetes.io/projected/5efde4bb-b7fb-4553-a3bc-2977126ea8e0-kube-api-access-hxwvq\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:09.459254 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.459220 2574 generic.go:356] "Generic (PLEG): container finished" podID="5efde4bb-b7fb-4553-a3bc-2977126ea8e0" containerID="6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d" exitCode=0 Jul 06 01:32:09.459419 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.459265 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588565fd56-dv2kk" Jul 06 01:32:09.459419 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.459282 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588565fd56-dv2kk" event={"ID":"5efde4bb-b7fb-4553-a3bc-2977126ea8e0","Type":"ContainerDied","Data":"6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d"} Jul 06 01:32:09.459419 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.459327 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588565fd56-dv2kk" event={"ID":"5efde4bb-b7fb-4553-a3bc-2977126ea8e0","Type":"ContainerDied","Data":"937bf44decbf710375b4d4d7b7d2d6fed5ee9cb698c835d62400b1084d24b685"} Jul 06 01:32:09.459419 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.459353 2574 scope.go:117] "RemoveContainer" containerID="6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d" Jul 06 01:32:09.468719 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.468700 2574 scope.go:117] "RemoveContainer" containerID="6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d" Jul 06 01:32:09.469033 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:32:09.469014 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d\": container with ID starting with 6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d not found: ID does not exist" containerID="6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d" Jul 06 01:32:09.469096 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.469042 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d"} err="failed to get container status \"6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d\": rpc error: code = NotFound desc = could not find container \"6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d\": container with ID starting with 6be644d31525f214c279a0222e7c61a64ccfe858b5ae4bf4be19462a4e91bf0d not found: ID does not exist" Jul 06 01:32:09.481411 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.481385 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-588565fd56-dv2kk"] Jul 06 01:32:09.487646 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:09.487624 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-588565fd56-dv2kk"] Jul 06 01:32:10.380159 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:10.380130 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5efde4bb-b7fb-4553-a3bc-2977126ea8e0" path="/var/lib/kubelet/pods/5efde4bb-b7fb-4553-a3bc-2977126ea8e0/volumes" Jul 06 01:32:18.494960 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:18.494925 2574 generic.go:356] "Generic (PLEG): container finished" podID="066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc" containerID="507e030d0982827eafd22572a44aaff80156e1f4308f703463b89b5c9914ebda" exitCode=0 Jul 06 01:32:18.495321 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:18.494968 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" event={"ID":"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc","Type":"ContainerDied","Data":"507e030d0982827eafd22572a44aaff80156e1f4308f703463b89b5c9914ebda"} Jul 06 01:32:18.495321 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:18.495248 2574 scope.go:117] "RemoveContainer" containerID="507e030d0982827eafd22572a44aaff80156e1f4308f703463b89b5c9914ebda" Jul 06 01:32:19.500532 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:19.500499 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-vtffd" event={"ID":"066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc","Type":"ContainerStarted","Data":"ea18bc36e71d621d04926e0f2abea1e5cff53f3dc1a94ef533e6265060786b5c"} Jul 06 01:32:21.634161 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:21.634123 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-64cdcc98f-dvb9k" podUID="f255014a-02eb-4a0d-babf-37d10ed853bf" containerName="console" containerID="cri-o://3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa" gracePeriod=15 Jul 06 01:32:21.874589 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:21.874565 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:32:22.039717 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.039641 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wm9v\" (UniqueName: \"kubernetes.io/projected/f255014a-02eb-4a0d-babf-37d10ed853bf-kube-api-access-6wm9v\") pod \"f255014a-02eb-4a0d-babf-37d10ed853bf\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " Jul 06 01:32:22.039717 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.039682 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-oauth-serving-cert\") pod \"f255014a-02eb-4a0d-babf-37d10ed853bf\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " Jul 06 01:32:22.039918 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.039819 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-console-config\") pod \"f255014a-02eb-4a0d-babf-37d10ed853bf\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " Jul 06 01:32:22.039918 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.039891 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-service-ca\") pod \"f255014a-02eb-4a0d-babf-37d10ed853bf\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " Jul 06 01:32:22.039918 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.039910 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-serving-cert\") pod \"f255014a-02eb-4a0d-babf-37d10ed853bf\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " Jul 06 01:32:22.040061 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.039945 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-oauth-config\") pod \"f255014a-02eb-4a0d-babf-37d10ed853bf\" (UID: \"f255014a-02eb-4a0d-babf-37d10ed853bf\") " Jul 06 01:32:22.040174 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.040142 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f255014a-02eb-4a0d-babf-37d10ed853bf" (UID: "f255014a-02eb-4a0d-babf-37d10ed853bf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:22.040303 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.040185 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-console-config" (OuterVolumeSpecName: "console-config") pod "f255014a-02eb-4a0d-babf-37d10ed853bf" (UID: "f255014a-02eb-4a0d-babf-37d10ed853bf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:22.040303 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.040291 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-service-ca" (OuterVolumeSpecName: "service-ca") pod "f255014a-02eb-4a0d-babf-37d10ed853bf" (UID: "f255014a-02eb-4a0d-babf-37d10ed853bf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:32:22.042131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.042095 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f255014a-02eb-4a0d-babf-37d10ed853bf" (UID: "f255014a-02eb-4a0d-babf-37d10ed853bf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:32:22.042131 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.042112 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f255014a-02eb-4a0d-babf-37d10ed853bf" (UID: "f255014a-02eb-4a0d-babf-37d10ed853bf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:32:22.042262 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.042165 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f255014a-02eb-4a0d-babf-37d10ed853bf-kube-api-access-6wm9v" (OuterVolumeSpecName: "kube-api-access-6wm9v") pod "f255014a-02eb-4a0d-babf-37d10ed853bf" (UID: "f255014a-02eb-4a0d-babf-37d10ed853bf"). InnerVolumeSpecName "kube-api-access-6wm9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:32:22.140574 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.140547 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-service-ca\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:22.140574 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.140571 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-serving-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:22.140731 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.140585 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f255014a-02eb-4a0d-babf-37d10ed853bf-console-oauth-config\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:22.140731 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.140599 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wm9v\" (UniqueName: \"kubernetes.io/projected/f255014a-02eb-4a0d-babf-37d10ed853bf-kube-api-access-6wm9v\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:22.140731 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.140610 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-oauth-serving-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:22.140731 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.140623 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f255014a-02eb-4a0d-babf-37d10ed853bf-console-config\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:32:22.512762 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.512733 2574 generic.go:356] "Generic (PLEG): container finished" podID="f255014a-02eb-4a0d-babf-37d10ed853bf" containerID="3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa" exitCode=0 Jul 06 01:32:22.512939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.512779 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64cdcc98f-dvb9k" Jul 06 01:32:22.512939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.512803 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cdcc98f-dvb9k" event={"ID":"f255014a-02eb-4a0d-babf-37d10ed853bf","Type":"ContainerDied","Data":"3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa"} Jul 06 01:32:22.512939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.512840 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64cdcc98f-dvb9k" event={"ID":"f255014a-02eb-4a0d-babf-37d10ed853bf","Type":"ContainerDied","Data":"ea82c2f3270e4811eac90d44eb9d835cbb0a0f47b3680b9146c3557ce9d5b5ae"} Jul 06 01:32:22.512939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.512874 2574 scope.go:117] "RemoveContainer" containerID="3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa" Jul 06 01:32:22.520969 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.520953 2574 scope.go:117] "RemoveContainer" containerID="3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa" Jul 06 01:32:22.521201 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:32:22.521184 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa\": container with ID starting with 3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa not found: ID does not exist" containerID="3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa" Jul 06 01:32:22.521246 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.521208 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa"} err="failed to get container status \"3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa\": rpc error: code = NotFound desc = could not find container \"3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa\": container with ID starting with 3671be6c9b1cb2005b4f909e743ac9f36a0db7fadc6cafb4783c3929ed3bf6fa not found: ID does not exist" Jul 06 01:32:22.527400 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.527377 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64cdcc98f-dvb9k"] Jul 06 01:32:22.530201 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:22.530184 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64cdcc98f-dvb9k"] Jul 06 01:32:24.379950 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:24.379913 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f255014a-02eb-4a0d-babf-37d10ed853bf" path="/var/lib/kubelet/pods/f255014a-02eb-4a0d-babf-37d10ed853bf/volumes" Jul 06 01:32:54.722733 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.722703 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fc5b7db48-4l4m2"] Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723007 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dfb1195-1355-4f46-8711-fffcb72b9491" containerName="registry" Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723019 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dfb1195-1355-4f46-8711-fffcb72b9491" containerName="registry" Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723032 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5efde4bb-b7fb-4553-a3bc-2977126ea8e0" containerName="console" Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723037 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5efde4bb-b7fb-4553-a3bc-2977126ea8e0" containerName="console" Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723046 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f255014a-02eb-4a0d-babf-37d10ed853bf" containerName="console" Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723051 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f255014a-02eb-4a0d-babf-37d10ed853bf" containerName="console" Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723093 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5efde4bb-b7fb-4553-a3bc-2977126ea8e0" containerName="console" Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723103 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f255014a-02eb-4a0d-babf-37d10ed853bf" containerName="console" Jul 06 01:32:54.723155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.723110 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dfb1195-1355-4f46-8711-fffcb72b9491" containerName="registry" Jul 06 01:32:54.727512 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.727497 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.733432 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.733409 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 01:32:54.733848 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.733827 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-cgnmd\"" Jul 06 01:32:54.733970 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.733953 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 01:32:54.734070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.734049 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 01:32:54.734153 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.734073 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 01:32:54.734476 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.734458 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 01:32:54.734574 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.734530 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 01:32:54.734574 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.734459 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 01:32:54.746222 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.746201 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 01:32:54.747447 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.747428 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fc5b7db48-4l4m2"] Jul 06 01:32:54.783247 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.783222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-oauth-config\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.783358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.783264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-oauth-serving-cert\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.783358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.783299 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvjd8\" (UniqueName: \"kubernetes.io/projected/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-kube-api-access-bvjd8\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.783358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.783337 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-service-ca\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.783512 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.783399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-trusted-ca-bundle\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.783512 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.783451 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-serving-cert\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.783512 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.783480 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-config\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884015 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-trusted-ca-bundle\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884061 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-serving-cert\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884261 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884079 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-config\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884261 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-oauth-config\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884261 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884228 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-oauth-serving-cert\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884411 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvjd8\" (UniqueName: \"kubernetes.io/projected/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-kube-api-access-bvjd8\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884411 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-service-ca\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884827 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884805 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-config\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.884986 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884965 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-oauth-serving-cert\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.885033 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.884976 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-service-ca\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.885758 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.885739 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-trusted-ca-bundle\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.886575 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.886559 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-oauth-config\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.886767 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.886750 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-serving-cert\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:54.903224 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:54.903197 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvjd8\" (UniqueName: \"kubernetes.io/projected/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-kube-api-access-bvjd8\") pod \"console-5fc5b7db48-4l4m2\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:55.036425 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:55.036355 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:32:55.165664 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:55.165637 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fc5b7db48-4l4m2"] Jul 06 01:32:55.169047 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:32:55.169024 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e26b553_4e78_4bb8_b5bd_a7e9180aaa90.slice/crio-81a1c5cd360a4e48d3589e2dad7662e3d7b9e5950850193d0ba843012a3fcee1 WatchSource:0}: Error finding container 81a1c5cd360a4e48d3589e2dad7662e3d7b9e5950850193d0ba843012a3fcee1: Status 404 returned error can't find the container with id 81a1c5cd360a4e48d3589e2dad7662e3d7b9e5950850193d0ba843012a3fcee1 Jul 06 01:32:55.638822 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:55.638776 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fc5b7db48-4l4m2" event={"ID":"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90","Type":"ContainerStarted","Data":"e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e"} Jul 06 01:32:55.638822 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:55.638823 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fc5b7db48-4l4m2" event={"ID":"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90","Type":"ContainerStarted","Data":"81a1c5cd360a4e48d3589e2dad7662e3d7b9e5950850193d0ba843012a3fcee1"} Jul 06 01:32:55.658319 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:55.658277 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fc5b7db48-4l4m2" podStartSLOduration=1.658262339 podStartE2EDuration="1.658262339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:32:55.657369599 +0000 UTC m=+373.792660103" watchObservedRunningTime="2026-07-06 01:32:55.658262339 +0000 UTC m=+373.793552842" Jul 06 01:32:56.395110 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:56.395080 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:32:56.397478 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:56.397462 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0a348e4-0203-4974-b406-002dc6a05d58-metrics-certs\") pod \"network-metrics-daemon-cn8h2\" (UID: \"b0a348e4-0203-4974-b406-002dc6a05d58\") " pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:32:56.580215 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:56.580186 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qszpf\"" Jul 06 01:32:56.587819 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:56.587802 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cn8h2" Jul 06 01:32:56.708628 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:56.708600 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cn8h2"] Jul 06 01:32:56.712017 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:32:56.711988 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a348e4_0203_4974_b406_002dc6a05d58.slice/crio-27d59f821283622f2613e2ffdcdc939ef08d003e3b30167aa097dc054a684a82 WatchSource:0}: Error finding container 27d59f821283622f2613e2ffdcdc939ef08d003e3b30167aa097dc054a684a82: Status 404 returned error can't find the container with id 27d59f821283622f2613e2ffdcdc939ef08d003e3b30167aa097dc054a684a82 Jul 06 01:32:57.648447 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:57.648410 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cn8h2" event={"ID":"b0a348e4-0203-4974-b406-002dc6a05d58","Type":"ContainerStarted","Data":"27d59f821283622f2613e2ffdcdc939ef08d003e3b30167aa097dc054a684a82"} Jul 06 01:32:58.023241 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.023160 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:32:58.026829 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.026806 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.029666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.029024 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 01:32:58.029666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.029259 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 01:32:58.029666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.029281 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 01:32:58.029666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.029500 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 01:32:58.029666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.029550 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 01:32:58.029666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.029560 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 01:32:58.030018 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.029847 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 01:32:58.033387 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.030265 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 01:32:58.033387 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.030528 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h7cmf\"" Jul 06 01:32:58.035872 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.035825 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 01:32:58.039919 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.039900 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:32:58.109402 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109377 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109505 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109414 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109505 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109444 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/012564a6-7f9e-4886-a4e3-9b9d699b79a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109505 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109493 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/012564a6-7f9e-4886-a4e3-9b9d699b79a6-config-out\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109604 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109521 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109604 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109604 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109588 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012564a6-7f9e-4886-a4e3-9b9d699b79a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109699 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109608 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/012564a6-7f9e-4886-a4e3-9b9d699b79a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109699 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109623 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/012564a6-7f9e-4886-a4e3-9b9d699b79a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109699 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109656 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6hcl\" (UniqueName: \"kubernetes.io/projected/012564a6-7f9e-4886-a4e3-9b9d699b79a6-kube-api-access-w6hcl\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109699 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109689 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-web-config\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109816 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109729 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.109816 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.109748 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.210948 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.210911 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/012564a6-7f9e-4886-a4e3-9b9d699b79a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.210948 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.210952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/012564a6-7f9e-4886-a4e3-9b9d699b79a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.210968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6hcl\" (UniqueName: \"kubernetes.io/projected/012564a6-7f9e-4886-a4e3-9b9d699b79a6-kube-api-access-w6hcl\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.210988 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-web-config\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211009 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211035 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211064 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211131 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/012564a6-7f9e-4886-a4e3-9b9d699b79a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211223 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/012564a6-7f9e-4886-a4e3-9b9d699b79a6-config-out\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211311 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012564a6-7f9e-4886-a4e3-9b9d699b79a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.211640 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.211586 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/012564a6-7f9e-4886-a4e3-9b9d699b79a6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.212284 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.212103 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/012564a6-7f9e-4886-a4e3-9b9d699b79a6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.214795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.212633 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/012564a6-7f9e-4886-a4e3-9b9d699b79a6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.214795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.214172 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/012564a6-7f9e-4886-a4e3-9b9d699b79a6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.214795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.214285 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-web-config\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.214795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.214711 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.215351 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.215242 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.215547 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.215530 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.215616 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.215554 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-config-volume\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.215675 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.215654 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.215735 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.215716 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/012564a6-7f9e-4886-a4e3-9b9d699b79a6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.216261 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.216244 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/012564a6-7f9e-4886-a4e3-9b9d699b79a6-config-out\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.218869 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.218832 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6hcl\" (UniqueName: \"kubernetes.io/projected/012564a6-7f9e-4886-a4e3-9b9d699b79a6-kube-api-access-w6hcl\") pod \"alertmanager-main-0\" (UID: \"012564a6-7f9e-4886-a4e3-9b9d699b79a6\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.340229 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.340203 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 01:32:58.473979 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.473946 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 01:32:58.475235 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:32:58.475210 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012564a6_7f9e_4886_a4e3_9b9d699b79a6.slice/crio-4397b4aa72a2b5c3ae2ad8e77a1e50038d224b9ee8a9ce919550587fc9b01df3 WatchSource:0}: Error finding container 4397b4aa72a2b5c3ae2ad8e77a1e50038d224b9ee8a9ce919550587fc9b01df3: Status 404 returned error can't find the container with id 4397b4aa72a2b5c3ae2ad8e77a1e50038d224b9ee8a9ce919550587fc9b01df3 Jul 06 01:32:58.654680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.654607 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cn8h2" event={"ID":"b0a348e4-0203-4974-b406-002dc6a05d58","Type":"ContainerStarted","Data":"90e27299ebde71120f7bf37e85100afcbfff15ffd42152084a281e9a755e6181"} Jul 06 01:32:58.654680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.654671 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cn8h2" event={"ID":"b0a348e4-0203-4974-b406-002dc6a05d58","Type":"ContainerStarted","Data":"17818ecd8bd01cb88ef8110e5eb251d5d3d2bb9c122b65dbbb49b37d60cdf0e3"} Jul 06 01:32:58.656073 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.656053 2574 generic.go:356] "Generic (PLEG): container finished" podID="012564a6-7f9e-4886-a4e3-9b9d699b79a6" containerID="f21eccd2beef1ff6b91d51bf5fa443ee7e4d2af56516552a6e89a3bb017f484b" exitCode=0 Jul 06 01:32:58.656172 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.656110 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"012564a6-7f9e-4886-a4e3-9b9d699b79a6","Type":"ContainerDied","Data":"f21eccd2beef1ff6b91d51bf5fa443ee7e4d2af56516552a6e89a3bb017f484b"} Jul 06 01:32:58.656172 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.656155 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"012564a6-7f9e-4886-a4e3-9b9d699b79a6","Type":"ContainerStarted","Data":"4397b4aa72a2b5c3ae2ad8e77a1e50038d224b9ee8a9ce919550587fc9b01df3"} Jul 06 01:32:58.669887 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:32:58.669816 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cn8h2" podStartSLOduration=375.779080332 podStartE2EDuration="6m16.669797074s" totalImagesPullingTime="890.716742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:32:58.668680904 +0000 UTC m=+376.803971409" watchObservedRunningTime="2026-07-06 01:32:58.669797074 +0000 UTC m=+376.805087580" Jul 06 01:33:00.667080 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:00.667048 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"012564a6-7f9e-4886-a4e3-9b9d699b79a6","Type":"ContainerStarted","Data":"e26dbc4569510b15ffef55ef46557169ad04185a14dda999a47380ca1b08f75b"} Jul 06 01:33:00.667365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:00.667111 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"012564a6-7f9e-4886-a4e3-9b9d699b79a6","Type":"ContainerStarted","Data":"0f06b267eb36e2cf1f744eabbcdb5ac322483cb5c6ad13779633d840ed281eda"} Jul 06 01:33:00.667365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:00.667137 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"012564a6-7f9e-4886-a4e3-9b9d699b79a6","Type":"ContainerStarted","Data":"1dbf819c90ce689610026c29fd9237fc9330f182f342db83ea5dc2cda442c2bd"} Jul 06 01:33:00.667365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:00.667161 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"012564a6-7f9e-4886-a4e3-9b9d699b79a6","Type":"ContainerStarted","Data":"e5af8b806adc850e2553a2ba8c06c1bdcf188c798644f5f93732ca37cfcbb772"} Jul 06 01:33:00.667365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:00.667185 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"012564a6-7f9e-4886-a4e3-9b9d699b79a6","Type":"ContainerStarted","Data":"74916ff702ca8f8b6c07cd656d0c87c97093d69c5fffa2149c5186ef1fa39504"} Jul 06 01:33:00.667365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:00.667208 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"012564a6-7f9e-4886-a4e3-9b9d699b79a6","Type":"ContainerStarted","Data":"75b2fd20daeaf1b116cd0c9de9cfe8e0e3ffb1edba4bd311b8daf59d789ffc18"} Jul 06 01:33:00.698560 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:00.698517 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.074342764 podStartE2EDuration="2.698502377s" totalImagesPullingTime="1.608407335s" totalInitContainerRuntime="15.752278ms" isStatefulPod=false podCreationTimestamp="2026-07-06 01:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:33:00.697352001 +0000 UTC m=+378.832642505" watchObservedRunningTime="2026-07-06 01:33:00.698502377 +0000 UTC m=+378.833792873" Jul 06 01:33:05.037325 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:05.037288 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:33:05.037697 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:05.037349 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:33:05.041975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:05.041956 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:33:05.690969 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:33:05.690942 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:34:04.337120 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.337089 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4"] Jul 06 01:34:04.340594 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.340572 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.342737 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.342716 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 01:34:04.342894 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.342761 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sqdgm\"" Jul 06 01:34:04.342894 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.342774 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 01:34:04.349700 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.349675 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4"] Jul 06 01:34:04.431847 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.431805 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.432043 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.431871 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.432043 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.431906 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qbxx\" (UniqueName: \"kubernetes.io/projected/6ad6595f-a021-496b-9066-ed0444818cab-kube-api-access-2qbxx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.532462 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.532378 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.532462 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.532420 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qbxx\" (UniqueName: \"kubernetes.io/projected/6ad6595f-a021-496b-9066-ed0444818cab-kube-api-access-2qbxx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.532712 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.532498 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.532772 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.532737 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.532897 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.532845 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.540626 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.540602 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qbxx\" (UniqueName: \"kubernetes.io/projected/6ad6595f-a021-496b-9066-ed0444818cab-kube-api-access-2qbxx\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.649837 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.649811 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:04.797179 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.797100 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4"] Jul 06 01:34:04.800017 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:34:04.799984 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad6595f_a021_496b_9066_ed0444818cab.slice/crio-f418ce303bc569c8542cecef6af0f1790d372229839ebcee4392fe887013aa15 WatchSource:0}: Error finding container f418ce303bc569c8542cecef6af0f1790d372229839ebcee4392fe887013aa15: Status 404 returned error can't find the container with id f418ce303bc569c8542cecef6af0f1790d372229839ebcee4392fe887013aa15 Jul 06 01:34:04.910173 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:04.910143 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" event={"ID":"6ad6595f-a021-496b-9066-ed0444818cab","Type":"ContainerStarted","Data":"f418ce303bc569c8542cecef6af0f1790d372229839ebcee4392fe887013aa15"} Jul 06 01:34:11.939570 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:11.939542 2574 generic.go:356] "Generic (PLEG): container finished" podID="6ad6595f-a021-496b-9066-ed0444818cab" containerID="2c63f38340242edd41d7cd53b1b52ec97b89c10e5c52d01d05563a4d4541e6f5" exitCode=0 Jul 06 01:34:11.939954 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:11.939621 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" event={"ID":"6ad6595f-a021-496b-9066-ed0444818cab","Type":"ContainerDied","Data":"2c63f38340242edd41d7cd53b1b52ec97b89c10e5c52d01d05563a4d4541e6f5"} Jul 06 01:34:14.953623 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:14.953589 2574 generic.go:356] "Generic (PLEG): container finished" podID="6ad6595f-a021-496b-9066-ed0444818cab" containerID="c774a225a0fceb22b65a4248894db675ae248aa7374a4d19d0493eb8a429b466" exitCode=0 Jul 06 01:34:14.954018 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:14.953656 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" event={"ID":"6ad6595f-a021-496b-9066-ed0444818cab","Type":"ContainerDied","Data":"c774a225a0fceb22b65a4248894db675ae248aa7374a4d19d0493eb8a429b466"} Jul 06 01:34:22.989002 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:22.988970 2574 generic.go:356] "Generic (PLEG): container finished" podID="6ad6595f-a021-496b-9066-ed0444818cab" containerID="919371b4ec5b0300ba9ecfa9cfcf4c02b7dee79922ce7c13dc2b5fc04d655f07" exitCode=0 Jul 06 01:34:22.989386 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:22.989042 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" event={"ID":"6ad6595f-a021-496b-9066-ed0444818cab","Type":"ContainerDied","Data":"919371b4ec5b0300ba9ecfa9cfcf4c02b7dee79922ce7c13dc2b5fc04d655f07"} Jul 06 01:34:24.127371 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.127343 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:24.299367 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.299278 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-util\") pod \"6ad6595f-a021-496b-9066-ed0444818cab\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " Jul 06 01:34:24.299528 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.299366 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-bundle\") pod \"6ad6595f-a021-496b-9066-ed0444818cab\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " Jul 06 01:34:24.299528 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.299392 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qbxx\" (UniqueName: \"kubernetes.io/projected/6ad6595f-a021-496b-9066-ed0444818cab-kube-api-access-2qbxx\") pod \"6ad6595f-a021-496b-9066-ed0444818cab\" (UID: \"6ad6595f-a021-496b-9066-ed0444818cab\") " Jul 06 01:34:24.300047 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.300018 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-bundle" (OuterVolumeSpecName: "bundle") pod "6ad6595f-a021-496b-9066-ed0444818cab" (UID: "6ad6595f-a021-496b-9066-ed0444818cab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:34:24.301488 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.301464 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad6595f-a021-496b-9066-ed0444818cab-kube-api-access-2qbxx" (OuterVolumeSpecName: "kube-api-access-2qbxx") pod "6ad6595f-a021-496b-9066-ed0444818cab" (UID: "6ad6595f-a021-496b-9066-ed0444818cab"). InnerVolumeSpecName "kube-api-access-2qbxx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:34:24.303680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.303651 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-util" (OuterVolumeSpecName: "util") pod "6ad6595f-a021-496b-9066-ed0444818cab" (UID: "6ad6595f-a021-496b-9066-ed0444818cab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:34:24.400654 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.400629 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-util\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:34:24.400654 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.400651 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ad6595f-a021-496b-9066-ed0444818cab-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:34:24.400779 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.400660 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qbxx\" (UniqueName: \"kubernetes.io/projected/6ad6595f-a021-496b-9066-ed0444818cab-kube-api-access-2qbxx\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:34:24.999250 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.999226 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" Jul 06 01:34:24.999250 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.999241 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540cwdk4" event={"ID":"6ad6595f-a021-496b-9066-ed0444818cab","Type":"ContainerDied","Data":"f418ce303bc569c8542cecef6af0f1790d372229839ebcee4392fe887013aa15"} Jul 06 01:34:24.999445 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:24.999285 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f418ce303bc569c8542cecef6af0f1790d372229839ebcee4392fe887013aa15" Jul 06 01:34:32.310924 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.310892 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l"] Jul 06 01:34:32.311281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.311181 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad6595f-a021-496b-9066-ed0444818cab" containerName="util" Jul 06 01:34:32.311281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.311192 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad6595f-a021-496b-9066-ed0444818cab" containerName="util" Jul 06 01:34:32.311281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.311213 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad6595f-a021-496b-9066-ed0444818cab" containerName="pull" Jul 06 01:34:32.311281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.311218 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad6595f-a021-496b-9066-ed0444818cab" containerName="pull" Jul 06 01:34:32.311281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.311230 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad6595f-a021-496b-9066-ed0444818cab" containerName="extract" Jul 06 01:34:32.311281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.311236 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad6595f-a021-496b-9066-ed0444818cab" containerName="extract" Jul 06 01:34:32.311281 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.311280 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad6595f-a021-496b-9066-ed0444818cab" containerName="extract" Jul 06 01:34:32.314483 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.314465 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" Jul 06 01:34:32.316562 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.316536 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 01:34:32.316670 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.316583 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 01:34:32.316670 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.316627 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jgql4\"" Jul 06 01:34:32.327025 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.326998 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l"] Jul 06 01:34:32.363321 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.363297 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlf97\" (UniqueName: \"kubernetes.io/projected/4b42502e-40b2-4e58-bfe4-d3a2a90b2499-kube-api-access-nlf97\") pod \"cert-manager-operator-controller-manager-55c444bf86-lsg7l\" (UID: \"4b42502e-40b2-4e58-bfe4-d3a2a90b2499\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" Jul 06 01:34:32.363421 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.363340 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b42502e-40b2-4e58-bfe4-d3a2a90b2499-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lsg7l\" (UID: \"4b42502e-40b2-4e58-bfe4-d3a2a90b2499\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" Jul 06 01:34:32.463708 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.463674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlf97\" (UniqueName: \"kubernetes.io/projected/4b42502e-40b2-4e58-bfe4-d3a2a90b2499-kube-api-access-nlf97\") pod \"cert-manager-operator-controller-manager-55c444bf86-lsg7l\" (UID: \"4b42502e-40b2-4e58-bfe4-d3a2a90b2499\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" Jul 06 01:34:32.463884 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.463760 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b42502e-40b2-4e58-bfe4-d3a2a90b2499-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lsg7l\" (UID: \"4b42502e-40b2-4e58-bfe4-d3a2a90b2499\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" Jul 06 01:34:32.464215 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.464180 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b42502e-40b2-4e58-bfe4-d3a2a90b2499-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lsg7l\" (UID: \"4b42502e-40b2-4e58-bfe4-d3a2a90b2499\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" Jul 06 01:34:32.472282 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.472261 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlf97\" (UniqueName: \"kubernetes.io/projected/4b42502e-40b2-4e58-bfe4-d3a2a90b2499-kube-api-access-nlf97\") pod \"cert-manager-operator-controller-manager-55c444bf86-lsg7l\" (UID: \"4b42502e-40b2-4e58-bfe4-d3a2a90b2499\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" Jul 06 01:34:32.623645 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.623621 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" Jul 06 01:34:32.751182 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:32.751158 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l"] Jul 06 01:34:32.753471 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:34:32.753446 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b42502e_40b2_4e58_bfe4_d3a2a90b2499.slice/crio-0be60cf601a2e979448d2faa8620314afe385912783eb196b64178992f2d8af0 WatchSource:0}: Error finding container 0be60cf601a2e979448d2faa8620314afe385912783eb196b64178992f2d8af0: Status 404 returned error can't find the container with id 0be60cf601a2e979448d2faa8620314afe385912783eb196b64178992f2d8af0 Jul 06 01:34:33.035892 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:33.035775 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" event={"ID":"4b42502e-40b2-4e58-bfe4-d3a2a90b2499","Type":"ContainerStarted","Data":"0be60cf601a2e979448d2faa8620314afe385912783eb196b64178992f2d8af0"} Jul 06 01:34:35.047050 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.047013 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" event={"ID":"4b42502e-40b2-4e58-bfe4-d3a2a90b2499","Type":"ContainerStarted","Data":"9a5adff153027d837c578cb8bb4ce2fc37d96b75ca31a8e80079f53199ef6bd0"} Jul 06 01:34:35.067157 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.067113 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lsg7l" podStartSLOduration=0.865252709 podStartE2EDuration="3.067099175s" totalImagesPullingTime="2.201846466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:34:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:34:35.064748275 +0000 UTC m=+473.200038780" watchObservedRunningTime="2026-07-06 01:34:35.067099175 +0000 UTC m=+473.202389680" Jul 06 01:34:35.920708 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.920675 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w9xmg"] Jul 06 01:34:35.923884 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.923847 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:35.925902 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.925879 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 01:34:35.925999 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.925902 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vd664\"" Jul 06 01:34:35.926309 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.926295 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 01:34:35.931276 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.931255 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w9xmg"] Jul 06 01:34:35.990904 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.990880 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6f98909-6c5a-4f37-b5e5-c231d0ab13a2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w9xmg\" (UID: \"e6f98909-6c5a-4f37-b5e5-c231d0ab13a2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:35.991015 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:35.990913 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmdl8\" (UniqueName: \"kubernetes.io/projected/e6f98909-6c5a-4f37-b5e5-c231d0ab13a2-kube-api-access-mmdl8\") pod \"cert-manager-webhook-6486ffd997-w9xmg\" (UID: \"e6f98909-6c5a-4f37-b5e5-c231d0ab13a2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:36.062850 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.062825 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq"] Jul 06 01:34:36.066087 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.066071 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.068090 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.068065 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 01:34:36.068184 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.068131 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 01:34:36.068184 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.068131 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sqdgm\"" Jul 06 01:34:36.073909 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.073889 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq"] Jul 06 01:34:36.092268 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.092245 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6f98909-6c5a-4f37-b5e5-c231d0ab13a2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w9xmg\" (UID: \"e6f98909-6c5a-4f37-b5e5-c231d0ab13a2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:36.092355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.092279 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmdl8\" (UniqueName: \"kubernetes.io/projected/e6f98909-6c5a-4f37-b5e5-c231d0ab13a2-kube-api-access-mmdl8\") pod \"cert-manager-webhook-6486ffd997-w9xmg\" (UID: \"e6f98909-6c5a-4f37-b5e5-c231d0ab13a2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:36.099455 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.099434 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6f98909-6c5a-4f37-b5e5-c231d0ab13a2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-w9xmg\" (UID: \"e6f98909-6c5a-4f37-b5e5-c231d0ab13a2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:36.099624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.099605 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmdl8\" (UniqueName: \"kubernetes.io/projected/e6f98909-6c5a-4f37-b5e5-c231d0ab13a2-kube-api-access-mmdl8\") pod \"cert-manager-webhook-6486ffd997-w9xmg\" (UID: \"e6f98909-6c5a-4f37-b5e5-c231d0ab13a2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:36.192824 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.192756 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.192824 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.192800 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.193010 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.192935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rk2z\" (UniqueName: \"kubernetes.io/projected/7b521e55-2ad3-4b72-8afd-86e696cd6412-kube-api-access-2rk2z\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.233652 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.233630 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:36.294149 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.294117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rk2z\" (UniqueName: \"kubernetes.io/projected/7b521e55-2ad3-4b72-8afd-86e696cd6412-kube-api-access-2rk2z\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.294306 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.294172 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.294306 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.294210 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.294712 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.294616 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.294712 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.294678 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.301936 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.301912 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rk2z\" (UniqueName: \"kubernetes.io/projected/7b521e55-2ad3-4b72-8afd-86e696cd6412-kube-api-access-2rk2z\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.359951 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.359875 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-w9xmg"] Jul 06 01:34:36.361909 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:34:36.361881 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6f98909_6c5a_4f37_b5e5_c231d0ab13a2.slice/crio-b57eb594e9885976027b6bb234ecbc725e7fd916178b89e9e2f5f19e54333114 WatchSource:0}: Error finding container b57eb594e9885976027b6bb234ecbc725e7fd916178b89e9e2f5f19e54333114: Status 404 returned error can't find the container with id b57eb594e9885976027b6bb234ecbc725e7fd916178b89e9e2f5f19e54333114 Jul 06 01:34:36.375413 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.375389 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:36.515420 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:36.515397 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq"] Jul 06 01:34:36.517687 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:34:36.517659 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b521e55_2ad3_4b72_8afd_86e696cd6412.slice/crio-40f6058e42359d76c9efd518216eeb9dc3efddf3618d36a3d188637bcfbd05b5 WatchSource:0}: Error finding container 40f6058e42359d76c9efd518216eeb9dc3efddf3618d36a3d188637bcfbd05b5: Status 404 returned error can't find the container with id 40f6058e42359d76c9efd518216eeb9dc3efddf3618d36a3d188637bcfbd05b5 Jul 06 01:34:37.060190 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:37.060154 2574 generic.go:356] "Generic (PLEG): container finished" podID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerID="ac64bcc9f45f450b5d5d1100b1c042dd34db61e801ce3ce70567e1366261e85e" exitCode=0 Jul 06 01:34:37.060376 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:37.060255 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" event={"ID":"7b521e55-2ad3-4b72-8afd-86e696cd6412","Type":"ContainerDied","Data":"ac64bcc9f45f450b5d5d1100b1c042dd34db61e801ce3ce70567e1366261e85e"} Jul 06 01:34:37.060376 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:37.060315 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" event={"ID":"7b521e55-2ad3-4b72-8afd-86e696cd6412","Type":"ContainerStarted","Data":"40f6058e42359d76c9efd518216eeb9dc3efddf3618d36a3d188637bcfbd05b5"} Jul 06 01:34:37.061499 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:37.061473 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" event={"ID":"e6f98909-6c5a-4f37-b5e5-c231d0ab13a2","Type":"ContainerStarted","Data":"b57eb594e9885976027b6bb234ecbc725e7fd916178b89e9e2f5f19e54333114"} Jul 06 01:34:39.496808 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.496772 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f26mg"] Jul 06 01:34:39.507441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.506565 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" Jul 06 01:34:39.508975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.508758 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mpffj\"" Jul 06 01:34:39.509329 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.509305 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f26mg"] Jul 06 01:34:39.521624 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.521574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gd6h\" (UniqueName: \"kubernetes.io/projected/f7fc43cc-31e3-41ca-a98a-937c81e25e9e-kube-api-access-2gd6h\") pod \"cert-manager-cainjector-559cdd8c87-f26mg\" (UID: \"f7fc43cc-31e3-41ca-a98a-937c81e25e9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" Jul 06 01:34:39.521818 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.521723 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7fc43cc-31e3-41ca-a98a-937c81e25e9e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f26mg\" (UID: \"f7fc43cc-31e3-41ca-a98a-937c81e25e9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" Jul 06 01:34:39.623176 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.623135 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7fc43cc-31e3-41ca-a98a-937c81e25e9e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f26mg\" (UID: \"f7fc43cc-31e3-41ca-a98a-937c81e25e9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" Jul 06 01:34:39.623334 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.623279 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gd6h\" (UniqueName: \"kubernetes.io/projected/f7fc43cc-31e3-41ca-a98a-937c81e25e9e-kube-api-access-2gd6h\") pod \"cert-manager-cainjector-559cdd8c87-f26mg\" (UID: \"f7fc43cc-31e3-41ca-a98a-937c81e25e9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" Jul 06 01:34:39.631501 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.631464 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7fc43cc-31e3-41ca-a98a-937c81e25e9e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-f26mg\" (UID: \"f7fc43cc-31e3-41ca-a98a-937c81e25e9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" Jul 06 01:34:39.631768 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.631741 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gd6h\" (UniqueName: \"kubernetes.io/projected/f7fc43cc-31e3-41ca-a98a-937c81e25e9e-kube-api-access-2gd6h\") pod \"cert-manager-cainjector-559cdd8c87-f26mg\" (UID: \"f7fc43cc-31e3-41ca-a98a-937c81e25e9e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" Jul 06 01:34:39.818783 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.818761 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" Jul 06 01:34:39.960610 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:39.960576 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-f26mg"] Jul 06 01:34:39.982610 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:34:39.982585 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7fc43cc_31e3_41ca_a98a_937c81e25e9e.slice/crio-48966af866dd919ea6a2e5e9b1d4fdcec49fae099f468d0ea269494649dc0187 WatchSource:0}: Error finding container 48966af866dd919ea6a2e5e9b1d4fdcec49fae099f468d0ea269494649dc0187: Status 404 returned error can't find the container with id 48966af866dd919ea6a2e5e9b1d4fdcec49fae099f468d0ea269494649dc0187 Jul 06 01:34:40.076180 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:40.076147 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" event={"ID":"e6f98909-6c5a-4f37-b5e5-c231d0ab13a2","Type":"ContainerStarted","Data":"41ae1329bb0809f311e63f2b3e1c82ee67d0e7d642f55214bf5595c2cc49aa8b"} Jul 06 01:34:40.076314 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:40.076197 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:40.078266 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:40.078238 2574 generic.go:356] "Generic (PLEG): container finished" podID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerID="4344e7456e70824c95cf2a5a9d3512303571c804622ab700996c6f1c7b88855d" exitCode=0 Jul 06 01:34:40.078406 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:40.078315 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" event={"ID":"7b521e55-2ad3-4b72-8afd-86e696cd6412","Type":"ContainerDied","Data":"4344e7456e70824c95cf2a5a9d3512303571c804622ab700996c6f1c7b88855d"} Jul 06 01:34:40.080183 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:40.080154 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" event={"ID":"f7fc43cc-31e3-41ca-a98a-937c81e25e9e","Type":"ContainerStarted","Data":"b83c8a3e9231469f5f52b56ab9df683ec4c97e3130715bda08be7abb3be81f61"} Jul 06 01:34:40.080307 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:40.080190 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" event={"ID":"f7fc43cc-31e3-41ca-a98a-937c81e25e9e","Type":"ContainerStarted","Data":"48966af866dd919ea6a2e5e9b1d4fdcec49fae099f468d0ea269494649dc0187"} Jul 06 01:34:40.108840 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:40.108799 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" podStartSLOduration=1.717018326 podStartE2EDuration="5.108784026s" totalImagesPullingTime="3.3917657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:34:40.09023791 +0000 UTC m=+478.225528415" watchObservedRunningTime="2026-07-06 01:34:40.108784026 +0000 UTC m=+478.244074529" Jul 06 01:34:40.126274 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:40.126235 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-f26mg" podStartSLOduration=1.126219633 podStartE2EDuration="1.126219633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:34:40.124489933 +0000 UTC m=+478.259780438" watchObservedRunningTime="2026-07-06 01:34:40.126219633 +0000 UTC m=+478.261510137" Jul 06 01:34:41.086446 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:41.086418 2574 generic.go:356] "Generic (PLEG): container finished" podID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerID="63da31d029beb358d4517718cfbb8fcfe4d2687189344b1d2c898d4ad6f533b8" exitCode=0 Jul 06 01:34:41.086803 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:41.086495 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" event={"ID":"7b521e55-2ad3-4b72-8afd-86e696cd6412","Type":"ContainerDied","Data":"63da31d029beb358d4517718cfbb8fcfe4d2687189344b1d2c898d4ad6f533b8"} Jul 06 01:34:42.238949 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.238922 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:42.346780 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.346709 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rk2z\" (UniqueName: \"kubernetes.io/projected/7b521e55-2ad3-4b72-8afd-86e696cd6412-kube-api-access-2rk2z\") pod \"7b521e55-2ad3-4b72-8afd-86e696cd6412\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " Jul 06 01:34:42.346780 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.346747 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-util\") pod \"7b521e55-2ad3-4b72-8afd-86e696cd6412\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " Jul 06 01:34:42.346965 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.346782 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-bundle\") pod \"7b521e55-2ad3-4b72-8afd-86e696cd6412\" (UID: \"7b521e55-2ad3-4b72-8afd-86e696cd6412\") " Jul 06 01:34:42.347253 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.347224 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-bundle" (OuterVolumeSpecName: "bundle") pod "7b521e55-2ad3-4b72-8afd-86e696cd6412" (UID: "7b521e55-2ad3-4b72-8afd-86e696cd6412"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:34:42.348922 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.348891 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b521e55-2ad3-4b72-8afd-86e696cd6412-kube-api-access-2rk2z" (OuterVolumeSpecName: "kube-api-access-2rk2z") pod "7b521e55-2ad3-4b72-8afd-86e696cd6412" (UID: "7b521e55-2ad3-4b72-8afd-86e696cd6412"). InnerVolumeSpecName "kube-api-access-2rk2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:34:42.351160 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.351129 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-util" (OuterVolumeSpecName: "util") pod "7b521e55-2ad3-4b72-8afd-86e696cd6412" (UID: "7b521e55-2ad3-4b72-8afd-86e696cd6412"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:34:42.447425 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.447400 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rk2z\" (UniqueName: \"kubernetes.io/projected/7b521e55-2ad3-4b72-8afd-86e696cd6412-kube-api-access-2rk2z\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:34:42.447425 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.447424 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-util\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:34:42.447598 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:42.447434 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b521e55-2ad3-4b72-8afd-86e696cd6412-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:34:43.097172 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:43.097129 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" event={"ID":"7b521e55-2ad3-4b72-8afd-86e696cd6412","Type":"ContainerDied","Data":"40f6058e42359d76c9efd518216eeb9dc3efddf3618d36a3d188637bcfbd05b5"} Jul 06 01:34:43.097336 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:43.097195 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40f6058e42359d76c9efd518216eeb9dc3efddf3618d36a3d188637bcfbd05b5" Jul 06 01:34:43.097336 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:43.097148 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fm4fsq" Jul 06 01:34:46.088775 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:46.088745 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-w9xmg" Jul 06 01:34:56.070180 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.070149 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt"] Jul 06 01:34:56.070642 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.070602 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerName="util" Jul 06 01:34:56.070642 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.070618 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerName="util" Jul 06 01:34:56.070642 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.070633 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerName="pull" Jul 06 01:34:56.070642 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.070641 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerName="pull" Jul 06 01:34:56.070883 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.070668 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerName="extract" Jul 06 01:34:56.070883 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.070677 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerName="extract" Jul 06 01:34:56.070883 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.070760 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b521e55-2ad3-4b72-8afd-86e696cd6412" containerName="extract" Jul 06 01:34:56.073711 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.073692 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.075659 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.075640 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sqdgm\"" Jul 06 01:34:56.075780 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.075665 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 01:34:56.075780 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.075668 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 01:34:56.080609 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.080587 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt"] Jul 06 01:34:56.164682 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.164658 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.164799 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.164697 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftrqq\" (UniqueName: \"kubernetes.io/projected/15d1e02e-4d20-490d-ab3e-f950f81232e5-kube-api-access-ftrqq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.164799 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.164728 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.265408 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.265383 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.265506 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.265429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftrqq\" (UniqueName: \"kubernetes.io/projected/15d1e02e-4d20-490d-ab3e-f950f81232e5-kube-api-access-ftrqq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.265506 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.265472 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.265824 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.265806 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.265885 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.265836 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.273079 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.273053 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftrqq\" (UniqueName: \"kubernetes.io/projected/15d1e02e-4d20-490d-ab3e-f950f81232e5-kube-api-access-ftrqq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.384009 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.383957 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:34:56.510160 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:56.510137 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt"] Jul 06 01:34:56.512556 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:34:56.512530 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d1e02e_4d20_490d_ab3e_f950f81232e5.slice/crio-23e40d46d63c4e8c5ba171df325e224a990f007e8ab50bb754123dbcbd79f06f WatchSource:0}: Error finding container 23e40d46d63c4e8c5ba171df325e224a990f007e8ab50bb754123dbcbd79f06f: Status 404 returned error can't find the container with id 23e40d46d63c4e8c5ba171df325e224a990f007e8ab50bb754123dbcbd79f06f Jul 06 01:34:57.161008 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:57.160976 2574 generic.go:356] "Generic (PLEG): container finished" podID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerID="98fa38f1f71c82ee7b1bc6fc12acec3c242e1736c071f18ff9e75c46797db892" exitCode=0 Jul 06 01:34:57.161378 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:57.161066 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" event={"ID":"15d1e02e-4d20-490d-ab3e-f950f81232e5","Type":"ContainerDied","Data":"98fa38f1f71c82ee7b1bc6fc12acec3c242e1736c071f18ff9e75c46797db892"} Jul 06 01:34:57.161378 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:57.161114 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" event={"ID":"15d1e02e-4d20-490d-ab3e-f950f81232e5","Type":"ContainerStarted","Data":"23e40d46d63c4e8c5ba171df325e224a990f007e8ab50bb754123dbcbd79f06f"} Jul 06 01:34:58.167757 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:58.167726 2574 generic.go:356] "Generic (PLEG): container finished" podID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerID="80d0ac5f1a2690b4eb14f9ed28443b38dc14efc53050faa02be08055cbff332e" exitCode=0 Jul 06 01:34:58.167757 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:58.167763 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" event={"ID":"15d1e02e-4d20-490d-ab3e-f950f81232e5","Type":"ContainerDied","Data":"80d0ac5f1a2690b4eb14f9ed28443b38dc14efc53050faa02be08055cbff332e"} Jul 06 01:34:59.174403 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:59.174369 2574 generic.go:356] "Generic (PLEG): container finished" podID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerID="cd91a5432e919013acbf604c6e498aa1cc2c2b04d6b0d418fa56e1f2842a4fec" exitCode=0 Jul 06 01:34:59.174763 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:34:59.174456 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" event={"ID":"15d1e02e-4d20-490d-ab3e-f950f81232e5","Type":"ContainerDied","Data":"cd91a5432e919013acbf604c6e498aa1cc2c2b04d6b0d418fa56e1f2842a4fec"} Jul 06 01:35:00.300716 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.300696 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:35:00.396615 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.396594 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-bundle\") pod \"15d1e02e-4d20-490d-ab3e-f950f81232e5\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " Jul 06 01:35:00.396767 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.396630 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-util\") pod \"15d1e02e-4d20-490d-ab3e-f950f81232e5\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " Jul 06 01:35:00.396767 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.396685 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftrqq\" (UniqueName: \"kubernetes.io/projected/15d1e02e-4d20-490d-ab3e-f950f81232e5-kube-api-access-ftrqq\") pod \"15d1e02e-4d20-490d-ab3e-f950f81232e5\" (UID: \"15d1e02e-4d20-490d-ab3e-f950f81232e5\") " Jul 06 01:35:00.397407 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.397337 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-bundle" (OuterVolumeSpecName: "bundle") pod "15d1e02e-4d20-490d-ab3e-f950f81232e5" (UID: "15d1e02e-4d20-490d-ab3e-f950f81232e5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:35:00.398887 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.398839 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d1e02e-4d20-490d-ab3e-f950f81232e5-kube-api-access-ftrqq" (OuterVolumeSpecName: "kube-api-access-ftrqq") pod "15d1e02e-4d20-490d-ab3e-f950f81232e5" (UID: "15d1e02e-4d20-490d-ab3e-f950f81232e5"). InnerVolumeSpecName "kube-api-access-ftrqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:35:00.404940 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.404917 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-util" (OuterVolumeSpecName: "util") pod "15d1e02e-4d20-490d-ab3e-f950f81232e5" (UID: "15d1e02e-4d20-490d-ab3e-f950f81232e5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:35:00.497499 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.497434 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftrqq\" (UniqueName: \"kubernetes.io/projected/15d1e02e-4d20-490d-ab3e-f950f81232e5-kube-api-access-ftrqq\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:35:00.497499 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.497459 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:35:00.497499 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:00.497470 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15d1e02e-4d20-490d-ab3e-f950f81232e5-util\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:35:01.186446 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:01.186412 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" Jul 06 01:35:01.186611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:01.186445 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7t5xt" event={"ID":"15d1e02e-4d20-490d-ab3e-f950f81232e5","Type":"ContainerDied","Data":"23e40d46d63c4e8c5ba171df325e224a990f007e8ab50bb754123dbcbd79f06f"} Jul 06 01:35:01.186611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:01.186488 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23e40d46d63c4e8c5ba171df325e224a990f007e8ab50bb754123dbcbd79f06f" Jul 06 01:35:13.129953 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.129917 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt"] Jul 06 01:35:13.130409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.130245 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerName="extract" Jul 06 01:35:13.130409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.130256 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerName="extract" Jul 06 01:35:13.130409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.130273 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerName="util" Jul 06 01:35:13.130409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.130279 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerName="util" Jul 06 01:35:13.130409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.130285 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerName="pull" Jul 06 01:35:13.130409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.130291 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerName="pull" Jul 06 01:35:13.130409 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.130342 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d1e02e-4d20-490d-ab3e-f950f81232e5" containerName="extract" Jul 06 01:35:13.136651 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.136628 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.138946 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.138922 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 01:35:13.139076 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.139051 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 01:35:13.139263 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.139245 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jrd6l\"" Jul 06 01:35:13.139400 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.139384 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 01:35:13.139501 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.139471 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 01:35:13.145214 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.145195 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt"] Jul 06 01:35:13.305373 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.305342 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5439a382-34f1-4d8f-bdf3-48db422e2539-tmp\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.305557 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.305393 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzhz9\" (UniqueName: \"kubernetes.io/projected/5439a382-34f1-4d8f-bdf3-48db422e2539-kube-api-access-dzhz9\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.305557 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.305445 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5439a382-34f1-4d8f-bdf3-48db422e2539-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.305557 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.305468 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5439a382-34f1-4d8f-bdf3-48db422e2539-manifests\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.305557 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.305507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5439a382-34f1-4d8f-bdf3-48db422e2539-webhook-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.406502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.406426 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5439a382-34f1-4d8f-bdf3-48db422e2539-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.406502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.406461 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5439a382-34f1-4d8f-bdf3-48db422e2539-manifests\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.406502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.406495 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5439a382-34f1-4d8f-bdf3-48db422e2539-webhook-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.406718 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.406626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5439a382-34f1-4d8f-bdf3-48db422e2539-tmp\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.406718 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.406682 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzhz9\" (UniqueName: \"kubernetes.io/projected/5439a382-34f1-4d8f-bdf3-48db422e2539-kube-api-access-dzhz9\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.406948 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.406928 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5439a382-34f1-4d8f-bdf3-48db422e2539-manifests\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.409023 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.409001 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5439a382-34f1-4d8f-bdf3-48db422e2539-tmp\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.409172 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.409121 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5439a382-34f1-4d8f-bdf3-48db422e2539-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.409225 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.409194 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5439a382-34f1-4d8f-bdf3-48db422e2539-webhook-cert\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.416114 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.416092 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzhz9\" (UniqueName: \"kubernetes.io/projected/5439a382-34f1-4d8f-bdf3-48db422e2539-kube-api-access-dzhz9\") pod \"opendatahub-operator-controller-manager-656987b587-lx7xt\" (UID: \"5439a382-34f1-4d8f-bdf3-48db422e2539\") " pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.448078 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.448044 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:13.585780 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:13.585716 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt"] Jul 06 01:35:13.588782 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:35:13.588756 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5439a382_34f1_4d8f_bdf3_48db422e2539.slice/crio-7447a635d33789775a68731f15fdea2210a41d1b6d88bcc98079d6cbd72f7ba8 WatchSource:0}: Error finding container 7447a635d33789775a68731f15fdea2210a41d1b6d88bcc98079d6cbd72f7ba8: Status 404 returned error can't find the container with id 7447a635d33789775a68731f15fdea2210a41d1b6d88bcc98079d6cbd72f7ba8 Jul 06 01:35:14.250108 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:14.250073 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" event={"ID":"5439a382-34f1-4d8f-bdf3-48db422e2539","Type":"ContainerStarted","Data":"7447a635d33789775a68731f15fdea2210a41d1b6d88bcc98079d6cbd72f7ba8"} Jul 06 01:35:17.268444 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:17.268413 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" event={"ID":"5439a382-34f1-4d8f-bdf3-48db422e2539","Type":"ContainerStarted","Data":"c0da9bf38010a29d1fbbdbfd91cb5f192c256c5f11b7a6aab01fb8e964f6864a"} Jul 06 01:35:18.274764 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:18.274721 2574 generic.go:356] "Generic (PLEG): container finished" podID="5439a382-34f1-4d8f-bdf3-48db422e2539" containerID="c0da9bf38010a29d1fbbdbfd91cb5f192c256c5f11b7a6aab01fb8e964f6864a" exitCode=0 Jul 06 01:35:18.275176 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:18.274813 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" event={"ID":"5439a382-34f1-4d8f-bdf3-48db422e2539","Type":"ContainerDied","Data":"c0da9bf38010a29d1fbbdbfd91cb5f192c256c5f11b7a6aab01fb8e964f6864a"} Jul 06 01:35:19.281185 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:19.281153 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" event={"ID":"5439a382-34f1-4d8f-bdf3-48db422e2539","Type":"ContainerStarted","Data":"8768d11ff8d8ba1f84cb6fece06ae4e543fe0253de389da9eb6c42f7304988d4"} Jul 06 01:35:19.281559 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:19.281344 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:19.303109 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:19.303063 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" podStartSLOduration=1.709446214 podStartE2EDuration="6.303048252s" totalImagesPullingTime="3.600760009s" totalInitContainerRuntime="992.842029ms" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:19.300532558 +0000 UTC m=+517.435823063" watchObservedRunningTime="2026-07-06 01:35:19.303048252 +0000 UTC m=+517.438338755" Jul 06 01:35:30.287680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:30.287642 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-656987b587-lx7xt" Jul 06 01:35:32.786717 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.786683 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4"] Jul 06 01:35:32.795968 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.795946 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.797846 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.797825 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vxsmh\"" Jul 06 01:35:32.798233 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.798220 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 01:35:32.798282 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.798249 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 01:35:32.805011 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.804986 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4"] Jul 06 01:35:32.865985 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.865957 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7qz2\" (UniqueName: \"kubernetes.io/projected/e570dbbc-7813-4a2b-b689-a208edf74ebc-kube-api-access-r7qz2\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.866093 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.866015 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e570dbbc-7813-4a2b-b689-a208edf74ebc-tls-certs\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.866093 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.866069 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e570dbbc-7813-4a2b-b689-a208edf74ebc-tmp\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.967354 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.967320 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7qz2\" (UniqueName: \"kubernetes.io/projected/e570dbbc-7813-4a2b-b689-a208edf74ebc-kube-api-access-r7qz2\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.967484 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.967436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e570dbbc-7813-4a2b-b689-a208edf74ebc-tls-certs\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.967484 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.967476 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e570dbbc-7813-4a2b-b689-a208edf74ebc-tmp\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.970249 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.970223 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e570dbbc-7813-4a2b-b689-a208edf74ebc-tmp\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.970559 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.970536 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e570dbbc-7813-4a2b-b689-a208edf74ebc-tls-certs\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:32.980801 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:32.980777 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7qz2\" (UniqueName: \"kubernetes.io/projected/e570dbbc-7813-4a2b-b689-a208edf74ebc-kube-api-access-r7qz2\") pod \"kube-auth-proxy-7f644fb7f6-ldnm4\" (UID: \"e570dbbc-7813-4a2b-b689-a208edf74ebc\") " pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:33.106548 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:33.106520 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" Jul 06 01:35:33.256609 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:33.256586 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4"] Jul 06 01:35:33.258784 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:35:33.258760 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode570dbbc_7813_4a2b_b689_a208edf74ebc.slice/crio-7a95b58d6e5b62f8de58ad6155267c5628694e5ec8b7bdb249ed4b6cd0d53c25 WatchSource:0}: Error finding container 7a95b58d6e5b62f8de58ad6155267c5628694e5ec8b7bdb249ed4b6cd0d53c25: Status 404 returned error can't find the container with id 7a95b58d6e5b62f8de58ad6155267c5628694e5ec8b7bdb249ed4b6cd0d53c25 Jul 06 01:35:33.342145 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:33.342111 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" event={"ID":"e570dbbc-7813-4a2b-b689-a208edf74ebc","Type":"ContainerStarted","Data":"7a95b58d6e5b62f8de58ad6155267c5628694e5ec8b7bdb249ed4b6cd0d53c25"} Jul 06 01:35:36.357604 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:36.357572 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" event={"ID":"e570dbbc-7813-4a2b-b689-a208edf74ebc","Type":"ContainerStarted","Data":"549e68d0fd1d4c8f5f2cd871c632472a0e9cbd788a157d91c9153c25ddb20e4d"} Jul 06 01:35:36.373307 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:36.373253 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f644fb7f6-ldnm4" podStartSLOduration=1.780360119 podStartE2EDuration="4.373233581s" totalImagesPullingTime="2.592873462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:36.372123903 +0000 UTC m=+534.507414408" watchObservedRunningTime="2026-07-06 01:35:36.373233581 +0000 UTC m=+534.508524087" Jul 06 01:35:40.525345 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.525313 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7"] Jul 06 01:35:40.528833 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.528817 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.532682 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.532653 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dfszh\"" Jul 06 01:35:40.533873 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.533842 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 01:35:40.551922 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.551901 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7"] Jul 06 01:35:40.633660 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.633637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.633785 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.633668 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b11c570f-2368-42f1-a601-af6747c34507-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.633785 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.633692 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.633785 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.633708 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.633990 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.633795 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.633990 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.633848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b11c570f-2368-42f1-a601-af6747c34507-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.633990 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.633919 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.633990 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.633967 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c92zd\" (UniqueName: \"kubernetes.io/projected/b11c570f-2368-42f1-a601-af6747c34507-kube-api-access-c92zd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.634124 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.634030 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b11c570f-2368-42f1-a601-af6747c34507-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735004 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.734972 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735165 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735008 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b11c570f-2368-42f1-a601-af6747c34507-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735165 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735033 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735165 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735057 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735165 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735085 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735165 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735105 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b11c570f-2368-42f1-a601-af6747c34507-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735165 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735129 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735510 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735167 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c92zd\" (UniqueName: \"kubernetes.io/projected/b11c570f-2368-42f1-a601-af6747c34507-kube-api-access-c92zd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735510 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735219 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b11c570f-2368-42f1-a601-af6747c34507-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735510 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735431 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735614 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735553 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735665 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735642 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735914 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735893 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.735979 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.735960 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b11c570f-2368-42f1-a601-af6747c34507-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.737901 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.737880 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b11c570f-2368-42f1-a601-af6747c34507-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.738112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.738094 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b11c570f-2368-42f1-a601-af6747c34507-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.745595 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.745573 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b11c570f-2368-42f1-a601-af6747c34507-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.745696 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.745672 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c92zd\" (UniqueName: \"kubernetes.io/projected/b11c570f-2368-42f1-a601-af6747c34507-kube-api-access-c92zd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46nlc7\" (UID: \"b11c570f-2368-42f1-a601-af6747c34507\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:40.839041 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:40.839012 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:41.002821 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:41.002796 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7"] Jul 06 01:35:41.005009 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:35:41.004984 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11c570f_2368_42f1_a601_af6747c34507.slice/crio-c6fe7d74542216e842fe14dbd7291ac94307b56383f7c8093435241770e2c755 WatchSource:0}: Error finding container c6fe7d74542216e842fe14dbd7291ac94307b56383f7c8093435241770e2c755: Status 404 returned error can't find the container with id c6fe7d74542216e842fe14dbd7291ac94307b56383f7c8093435241770e2c755 Jul 06 01:35:41.381991 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:41.381963 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" event={"ID":"b11c570f-2368-42f1-a601-af6747c34507","Type":"ContainerStarted","Data":"c6fe7d74542216e842fe14dbd7291ac94307b56383f7c8093435241770e2c755"} Jul 06 01:35:42.676412 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.676380 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7"] Jul 06 01:35:42.680257 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.680235 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.684493 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.684155 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 01:35:42.684493 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.684274 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7nxcc\"" Jul 06 01:35:42.684493 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.684371 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 01:35:42.685426 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.685291 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 01:35:42.694622 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.694594 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7"] Jul 06 01:35:42.756359 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.756324 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf5rw\" (UniqueName: \"kubernetes.io/projected/478f87cf-c161-4a86-b3dd-2d71dbf45de3-kube-api-access-gf5rw\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.756518 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.756391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.756518 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.756421 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.756518 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.756453 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.756652 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.756524 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.756652 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.756583 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/478f87cf-c161-4a86-b3dd-2d71dbf45de3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.756652 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.756646 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.857132 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.857097 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/478f87cf-c161-4a86-b3dd-2d71dbf45de3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.857314 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.857145 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.857314 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.857195 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf5rw\" (UniqueName: \"kubernetes.io/projected/478f87cf-c161-4a86-b3dd-2d71dbf45de3-kube-api-access-gf5rw\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.857314 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.857238 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.857314 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.857269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.857314 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.857304 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.857588 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.857346 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.857938 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.857897 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.859820 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.859789 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/478f87cf-c161-4a86-b3dd-2d71dbf45de3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.860007 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.859979 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.860094 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.860075 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.860139 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.860128 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.877514 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.877491 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf5rw\" (UniqueName: \"kubernetes.io/projected/478f87cf-c161-4a86-b3dd-2d71dbf45de3-kube-api-access-gf5rw\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.880546 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.880495 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/478f87cf-c161-4a86-b3dd-2d71dbf45de3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-stbx7\" (UID: \"478f87cf-c161-4a86-b3dd-2d71dbf45de3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:42.992163 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:42.992078 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:43.758542 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:43.758499 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:35:43.758941 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:43.758596 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:35:43.758941 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:43.758648 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:35:43.852385 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:35:43.852358 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478f87cf_c161_4a86_b3dd_2d71dbf45de3.slice/crio-e598e88e0036133e585a05c0742e358f86e63638ff369d36f1e4d95429d5eb72 WatchSource:0}: Error finding container e598e88e0036133e585a05c0742e358f86e63638ff369d36f1e4d95429d5eb72: Status 404 returned error can't find the container with id e598e88e0036133e585a05c0742e358f86e63638ff369d36f1e4d95429d5eb72 Jul 06 01:35:43.855917 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:43.855890 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7"] Jul 06 01:35:44.149763 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.149732 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l"] Jul 06 01:35:44.153355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.153332 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:44.155395 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.155372 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 01:35:44.155521 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.155445 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-l8jqt\"" Jul 06 01:35:44.163013 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.162978 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l"] Jul 06 01:35:44.271347 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.271316 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e7ae9b0-50f2-48f3-b07c-d5d58badcee7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f669l\" (UID: \"3e7ae9b0-50f2-48f3-b07c-d5d58badcee7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:44.271494 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.271355 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ftsk\" (UniqueName: \"kubernetes.io/projected/3e7ae9b0-50f2-48f3-b07c-d5d58badcee7-kube-api-access-5ftsk\") pod \"llmisvc-controller-manager-777c6bd7d5-f669l\" (UID: \"3e7ae9b0-50f2-48f3-b07c-d5d58badcee7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:44.372295 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.372263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e7ae9b0-50f2-48f3-b07c-d5d58badcee7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f669l\" (UID: \"3e7ae9b0-50f2-48f3-b07c-d5d58badcee7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:44.372479 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.372304 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ftsk\" (UniqueName: \"kubernetes.io/projected/3e7ae9b0-50f2-48f3-b07c-d5d58badcee7-kube-api-access-5ftsk\") pod \"llmisvc-controller-manager-777c6bd7d5-f669l\" (UID: \"3e7ae9b0-50f2-48f3-b07c-d5d58badcee7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:44.375000 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.374971 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e7ae9b0-50f2-48f3-b07c-d5d58badcee7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-f669l\" (UID: \"3e7ae9b0-50f2-48f3-b07c-d5d58badcee7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:44.399479 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.399452 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" event={"ID":"b11c570f-2368-42f1-a601-af6747c34507","Type":"ContainerStarted","Data":"c0ee1fc8c246efca02d4803727b5c0074685373661dcf7ae5cd34733e754cf02"} Jul 06 01:35:44.400710 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.400657 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" event={"ID":"478f87cf-c161-4a86-b3dd-2d71dbf45de3","Type":"ContainerStarted","Data":"e598e88e0036133e585a05c0742e358f86e63638ff369d36f1e4d95429d5eb72"} Jul 06 01:35:44.423380 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.423355 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ftsk\" (UniqueName: \"kubernetes.io/projected/3e7ae9b0-50f2-48f3-b07c-d5d58badcee7-kube-api-access-5ftsk\") pod \"llmisvc-controller-manager-777c6bd7d5-f669l\" (UID: \"3e7ae9b0-50f2-48f3-b07c-d5d58badcee7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:44.466683 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.466662 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:44.621851 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.621789 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" podStartSLOduration=1.8704754110000001 podStartE2EDuration="4.621763139s" totalImagesPullingTime="2.751287728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:44.462557909 +0000 UTC m=+542.597848414" watchObservedRunningTime="2026-07-06 01:35:44.621763139 +0000 UTC m=+542.757053646" Jul 06 01:35:44.624082 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.624056 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l"] Jul 06 01:35:44.839482 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.839436 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:44.846463 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:44.846435 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:45.407596 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:45.407555 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" event={"ID":"3e7ae9b0-50f2-48f3-b07c-d5d58badcee7","Type":"ContainerStarted","Data":"cc242c78de1c2dd7fe42428b52c891ea129ff295da9f887a2dff2406384d6fb7"} Jul 06 01:35:45.407960 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:45.407891 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:45.409364 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:45.409342 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46nlc7" Jul 06 01:35:46.660151 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:46.660107 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:35:46.660510 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:46.660192 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:35:47.423371 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:47.423296 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" event={"ID":"478f87cf-c161-4a86-b3dd-2d71dbf45de3","Type":"ContainerStarted","Data":"dcccc0bb28e7bf06f0c75c76c263f455a94dff7ee37bc3c175fd6141e3297268"} Jul 06 01:35:47.449920 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:47.449668 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" podStartSLOduration=2.645128049 podStartE2EDuration="5.449650787s" totalImagesPullingTime="2.804522738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:47.448693737 +0000 UTC m=+545.583984241" watchObservedRunningTime="2026-07-06 01:35:47.449650787 +0000 UTC m=+545.584941292" Jul 06 01:35:48.428594 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:48.428561 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:48.430605 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:48.430580 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-stbx7" Jul 06 01:35:49.434490 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:49.434452 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" event={"ID":"3e7ae9b0-50f2-48f3-b07c-d5d58badcee7","Type":"ContainerStarted","Data":"1f7a4edf29ff8275db352eab8f70fcfa1a7d06c006a69bd068ce9780e01367b0"} Jul 06 01:35:49.434913 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:49.434769 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:35:49.452038 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:35:49.451999 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" podStartSLOduration=1.419860548 podStartE2EDuration="5.451986285s" totalImagesPullingTime="4.032125737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:35:49.450299799 +0000 UTC m=+547.585590304" watchObservedRunningTime="2026-07-06 01:35:49.451986285 +0000 UTC m=+547.587276788" Jul 06 01:36:20.441849 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:20.441820 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-f669l" Jul 06 01:36:42.285274 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:42.285232 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:36:42.286741 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:42.286712 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:36:42.299155 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:42.299131 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:36:42.300092 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:42.300076 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:36:47.846037 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.846005 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm"] Jul 06 01:36:47.848483 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.848467 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:47.850533 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.850509 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 01:36:47.850954 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.850931 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cgz9t\"" Jul 06 01:36:47.851055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.850994 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 01:36:47.856565 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.856263 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm"] Jul 06 01:36:47.902212 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.902183 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:47.902363 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.902235 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg2w9\" (UniqueName: \"kubernetes.io/projected/5279d4cb-7a8c-4b59-a45a-ac16635c2216-kube-api-access-fg2w9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:47.902363 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:47.902267 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:48.003457 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.003429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg2w9\" (UniqueName: \"kubernetes.io/projected/5279d4cb-7a8c-4b59-a45a-ac16635c2216-kube-api-access-fg2w9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:48.003610 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.003472 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:48.003610 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.003523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:48.003847 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.003829 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:48.003959 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.003897 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:48.012227 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.012204 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg2w9\" (UniqueName: \"kubernetes.io/projected/5279d4cb-7a8c-4b59-a45a-ac16635c2216-kube-api-access-fg2w9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:48.159358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.159284 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:48.290200 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.290175 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm"] Jul 06 01:36:48.292428 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:36:48.292403 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5279d4cb_7a8c_4b59_a45a_ac16635c2216.slice/crio-5617974c16161cddffef297f15b1014efb65423e9ea0221fbf26ab1120be8651 WatchSource:0}: Error finding container 5617974c16161cddffef297f15b1014efb65423e9ea0221fbf26ab1120be8651: Status 404 returned error can't find the container with id 5617974c16161cddffef297f15b1014efb65423e9ea0221fbf26ab1120be8651 Jul 06 01:36:48.446071 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.446006 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw"] Jul 06 01:36:48.448601 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.448586 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.459531 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.459507 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw"] Jul 06 01:36:48.508590 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.508562 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.508766 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.508610 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.508766 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.508634 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsqgp\" (UniqueName: \"kubernetes.io/projected/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-kube-api-access-rsqgp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.609731 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.609702 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.609914 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.609735 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsqgp\" (UniqueName: \"kubernetes.io/projected/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-kube-api-access-rsqgp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.609914 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.609797 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.610120 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.610102 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.610169 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.610148 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.617402 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.617380 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsqgp\" (UniqueName: \"kubernetes.io/projected/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-kube-api-access-rsqgp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.716564 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.716481 2574 generic.go:356] "Generic (PLEG): container finished" podID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerID="8a9e9c39a4a82d1b3a1191db8b7ab54a296ea4a8f62addce9a34db03155a9314" exitCode=0 Jul 06 01:36:48.716705 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.716576 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" event={"ID":"5279d4cb-7a8c-4b59-a45a-ac16635c2216","Type":"ContainerDied","Data":"8a9e9c39a4a82d1b3a1191db8b7ab54a296ea4a8f62addce9a34db03155a9314"} Jul 06 01:36:48.716705 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.716617 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" event={"ID":"5279d4cb-7a8c-4b59-a45a-ac16635c2216","Type":"ContainerStarted","Data":"5617974c16161cddffef297f15b1014efb65423e9ea0221fbf26ab1120be8651"} Jul 06 01:36:48.717091 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.717077 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:36:48.757385 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.757360 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:48.883828 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:48.883799 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw"] Jul 06 01:36:48.886277 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:36:48.886241 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35365362_fd3d_4dcd_a422_1aa8acaa9dfb.slice/crio-fc7c89c810d141dde279285bd3d3a31bd16ab7221c65a39fe44a44f3bd194b7e WatchSource:0}: Error finding container fc7c89c810d141dde279285bd3d3a31bd16ab7221c65a39fe44a44f3bd194b7e: Status 404 returned error can't find the container with id fc7c89c810d141dde279285bd3d3a31bd16ab7221c65a39fe44a44f3bd194b7e Jul 06 01:36:49.045437 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.045406 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh"] Jul 06 01:36:49.048787 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.048769 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.055484 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.055453 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh"] Jul 06 01:36:49.115895 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.115838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.116047 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.115996 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kjvt\" (UniqueName: \"kubernetes.io/projected/81ed29d4-4135-429a-98cc-c562c1d3fabe-kube-api-access-8kjvt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.116104 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.116049 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.129125 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.129097 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d879dd9fb-xf54x"] Jul 06 01:36:49.132509 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.132485 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.143440 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.143419 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d879dd9fb-xf54x"] Jul 06 01:36:49.216639 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216606 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kjvt\" (UniqueName: \"kubernetes.io/projected/81ed29d4-4135-429a-98cc-c562c1d3fabe-kube-api-access-8kjvt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.216639 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216644 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-config\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.216842 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216661 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-trusted-ca-bundle\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.216842 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216678 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-oauth-serving-cert\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.216842 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216779 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.216842 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216811 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-serving-cert\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.217069 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216881 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.217069 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216911 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-service-ca\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.217069 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lnpf\" (UniqueName: \"kubernetes.io/projected/e9cb887b-1da4-4f0e-add3-7b239286f4e5-kube-api-access-5lnpf\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.217069 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.216964 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-oauth-config\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.217273 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.217200 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.217273 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.217258 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.224339 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.224316 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kjvt\" (UniqueName: \"kubernetes.io/projected/81ed29d4-4135-429a-98cc-c562c1d3fabe-kube-api-access-8kjvt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.318432 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.318361 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-config\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.318432 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.318395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-trusted-ca-bundle\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.318432 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.318411 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-oauth-serving-cert\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.318602 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.318542 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-serving-cert\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.318602 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.318587 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-service-ca\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.318680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.318620 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lnpf\" (UniqueName: \"kubernetes.io/projected/e9cb887b-1da4-4f0e-add3-7b239286f4e5-kube-api-access-5lnpf\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.318680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.318649 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-oauth-config\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.319212 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.319188 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-config\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.319328 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.319254 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-oauth-serving-cert\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.319328 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.319304 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-trusted-ca-bundle\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.319503 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.319484 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cb887b-1da4-4f0e-add3-7b239286f4e5-service-ca\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.321085 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.321061 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-serving-cert\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.321201 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.321183 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e9cb887b-1da4-4f0e-add3-7b239286f4e5-console-oauth-config\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.325937 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.325918 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lnpf\" (UniqueName: \"kubernetes.io/projected/e9cb887b-1da4-4f0e-add3-7b239286f4e5-kube-api-access-5lnpf\") pod \"console-6d879dd9fb-xf54x\" (UID: \"e9cb887b-1da4-4f0e-add3-7b239286f4e5\") " pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.358820 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.358792 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:49.444304 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.444270 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:49.491517 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.491399 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh"] Jul 06 01:36:49.530024 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:36:49.529981 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81ed29d4_4135_429a_98cc_c562c1d3fabe.slice/crio-c533664b578451845ec5c3922f097ea58e81cd8152cc775fc69bab9d66bad776 WatchSource:0}: Error finding container c533664b578451845ec5c3922f097ea58e81cd8152cc775fc69bab9d66bad776: Status 404 returned error can't find the container with id c533664b578451845ec5c3922f097ea58e81cd8152cc775fc69bab9d66bad776 Jul 06 01:36:49.585279 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.585238 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d879dd9fb-xf54x"] Jul 06 01:36:49.589332 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:36:49.589308 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9cb887b_1da4_4f0e_add3_7b239286f4e5.slice/crio-cf37ba7bb85e2c8c4e25d9adf01a4f3baa1023ccb045f879ffb3f58313a290e5 WatchSource:0}: Error finding container cf37ba7bb85e2c8c4e25d9adf01a4f3baa1023ccb045f879ffb3f58313a290e5: Status 404 returned error can't find the container with id cf37ba7bb85e2c8c4e25d9adf01a4f3baa1023ccb045f879ffb3f58313a290e5 Jul 06 01:36:49.650226 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.650200 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg"] Jul 06 01:36:49.658923 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.656429 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.662343 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.662321 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg"] Jul 06 01:36:49.722323 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.722296 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg7nc\" (UniqueName: \"kubernetes.io/projected/edf02925-4a13-423d-bc70-f477402fd30e-kube-api-access-dg7nc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.722508 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.722385 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.722508 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.722434 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.723649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.723624 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d879dd9fb-xf54x" event={"ID":"e9cb887b-1da4-4f0e-add3-7b239286f4e5","Type":"ContainerStarted","Data":"9b88905fbee577a3abec46379c27eeaa8c0ac6533444c841e0513f3ea5e76ab4"} Jul 06 01:36:49.723773 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.723675 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d879dd9fb-xf54x" event={"ID":"e9cb887b-1da4-4f0e-add3-7b239286f4e5","Type":"ContainerStarted","Data":"cf37ba7bb85e2c8c4e25d9adf01a4f3baa1023ccb045f879ffb3f58313a290e5"} Jul 06 01:36:49.725137 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.725114 2574 generic.go:356] "Generic (PLEG): container finished" podID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerID="716adbe616c3af8bf8d87c0517f6df993aa588570c1b6ade55dbdc890d9a8813" exitCode=0 Jul 06 01:36:49.725237 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.725191 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" event={"ID":"35365362-fd3d-4dcd-a422-1aa8acaa9dfb","Type":"ContainerDied","Data":"716adbe616c3af8bf8d87c0517f6df993aa588570c1b6ade55dbdc890d9a8813"} Jul 06 01:36:49.725237 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.725230 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" event={"ID":"35365362-fd3d-4dcd-a422-1aa8acaa9dfb","Type":"ContainerStarted","Data":"fc7c89c810d141dde279285bd3d3a31bd16ab7221c65a39fe44a44f3bd194b7e"} Jul 06 01:36:49.726837 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.726817 2574 generic.go:356] "Generic (PLEG): container finished" podID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerID="582c10debb82869d67416e9c8b2728ac16164bb340e7ee6768374329b5977527" exitCode=0 Jul 06 01:36:49.727049 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.726900 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" event={"ID":"81ed29d4-4135-429a-98cc-c562c1d3fabe","Type":"ContainerDied","Data":"582c10debb82869d67416e9c8b2728ac16164bb340e7ee6768374329b5977527"} Jul 06 01:36:49.727186 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.727169 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" event={"ID":"81ed29d4-4135-429a-98cc-c562c1d3fabe","Type":"ContainerStarted","Data":"c533664b578451845ec5c3922f097ea58e81cd8152cc775fc69bab9d66bad776"} Jul 06 01:36:49.729247 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.729224 2574 generic.go:356] "Generic (PLEG): container finished" podID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerID="a4f3f9fddf8ce85fb5ac93972a470edec5ac2fac249a2775fa4d3e7642819c1c" exitCode=0 Jul 06 01:36:49.729456 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.729260 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" event={"ID":"5279d4cb-7a8c-4b59-a45a-ac16635c2216","Type":"ContainerDied","Data":"a4f3f9fddf8ce85fb5ac93972a470edec5ac2fac249a2775fa4d3e7642819c1c"} Jul 06 01:36:49.741344 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.741304 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d879dd9fb-xf54x" podStartSLOduration=0.741291958 podStartE2EDuration="741.291958ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:36:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:36:49.738676121 +0000 UTC m=+607.873966626" watchObservedRunningTime="2026-07-06 01:36:49.741291958 +0000 UTC m=+607.876582517" Jul 06 01:36:49.823650 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.823620 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg7nc\" (UniqueName: \"kubernetes.io/projected/edf02925-4a13-423d-bc70-f477402fd30e-kube-api-access-dg7nc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.823777 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.823699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.823837 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.823790 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.824328 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.824303 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.824430 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.824383 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:49.831618 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:49.831561 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg7nc\" (UniqueName: \"kubernetes.io/projected/edf02925-4a13-423d-bc70-f477402fd30e-kube-api-access-dg7nc\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:50.034263 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.034235 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:50.165945 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.165921 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg"] Jul 06 01:36:50.167897 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:36:50.167842 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf02925_4a13_423d_bc70_f477402fd30e.slice/crio-083248f0f0d8cfa1c8cf7c0174de5e40be428f6f3656eb25cf1a20a7a3ce8549 WatchSource:0}: Error finding container 083248f0f0d8cfa1c8cf7c0174de5e40be428f6f3656eb25cf1a20a7a3ce8549: Status 404 returned error can't find the container with id 083248f0f0d8cfa1c8cf7c0174de5e40be428f6f3656eb25cf1a20a7a3ce8549 Jul 06 01:36:50.736225 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.736200 2574 generic.go:356] "Generic (PLEG): container finished" podID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerID="54b66ddf8bdce4c2cf68d985d11d6d0b7c49ac99479b2b96482e3f27bb59143f" exitCode=0 Jul 06 01:36:50.736339 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.736267 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" event={"ID":"35365362-fd3d-4dcd-a422-1aa8acaa9dfb","Type":"ContainerDied","Data":"54b66ddf8bdce4c2cf68d985d11d6d0b7c49ac99479b2b96482e3f27bb59143f"} Jul 06 01:36:50.737899 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.737830 2574 generic.go:356] "Generic (PLEG): container finished" podID="edf02925-4a13-423d-bc70-f477402fd30e" containerID="b54e9b22be1d35ea8b9cdff771dc9020fd34f94775beaf2ff24576cc0b7d6296" exitCode=0 Jul 06 01:36:50.737899 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.737889 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" event={"ID":"edf02925-4a13-423d-bc70-f477402fd30e","Type":"ContainerDied","Data":"b54e9b22be1d35ea8b9cdff771dc9020fd34f94775beaf2ff24576cc0b7d6296"} Jul 06 01:36:50.738039 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.737926 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" event={"ID":"edf02925-4a13-423d-bc70-f477402fd30e","Type":"ContainerStarted","Data":"083248f0f0d8cfa1c8cf7c0174de5e40be428f6f3656eb25cf1a20a7a3ce8549"} Jul 06 01:36:50.740064 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.739916 2574 generic.go:356] "Generic (PLEG): container finished" podID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerID="03d0ff3413fa36aec04ea2b1b5da1379195176a6876c96f317b75353e10bcdbf" exitCode=0 Jul 06 01:36:50.740064 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.740002 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" event={"ID":"81ed29d4-4135-429a-98cc-c562c1d3fabe","Type":"ContainerDied","Data":"03d0ff3413fa36aec04ea2b1b5da1379195176a6876c96f317b75353e10bcdbf"} Jul 06 01:36:50.742269 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.742246 2574 generic.go:356] "Generic (PLEG): container finished" podID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerID="f1c9aaf81dde8497ce7ba829ad0a596b1990d9e437323cd1fd9dd1e7b4c9bf8c" exitCode=0 Jul 06 01:36:50.742374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:50.742347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" event={"ID":"5279d4cb-7a8c-4b59-a45a-ac16635c2216","Type":"ContainerDied","Data":"f1c9aaf81dde8497ce7ba829ad0a596b1990d9e437323cd1fd9dd1e7b4c9bf8c"} Jul 06 01:36:51.749372 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:51.749343 2574 generic.go:356] "Generic (PLEG): container finished" podID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerID="a803e931459b5b782f9db17d45d7b9daf6090a3b4400fb3611fb57d59cf6059c" exitCode=0 Jul 06 01:36:51.749750 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:51.749427 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" event={"ID":"35365362-fd3d-4dcd-a422-1aa8acaa9dfb","Type":"ContainerDied","Data":"a803e931459b5b782f9db17d45d7b9daf6090a3b4400fb3611fb57d59cf6059c"} Jul 06 01:36:51.751270 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:51.751250 2574 generic.go:356] "Generic (PLEG): container finished" podID="edf02925-4a13-423d-bc70-f477402fd30e" containerID="c3de5b7ac2675dfb18dc3a5f6abbbe02a26ccfeffca0ed8180f2a3a309f5a438" exitCode=0 Jul 06 01:36:51.751369 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:51.751333 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" event={"ID":"edf02925-4a13-423d-bc70-f477402fd30e","Type":"ContainerDied","Data":"c3de5b7ac2675dfb18dc3a5f6abbbe02a26ccfeffca0ed8180f2a3a309f5a438"} Jul 06 01:36:51.753470 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:51.753451 2574 generic.go:356] "Generic (PLEG): container finished" podID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerID="3dfaa6a24eb71d1ada335bf6c649daad221b506b43dedadf03f3857d8a563f90" exitCode=0 Jul 06 01:36:51.753548 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:51.753516 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" event={"ID":"81ed29d4-4135-429a-98cc-c562c1d3fabe","Type":"ContainerDied","Data":"3dfaa6a24eb71d1ada335bf6c649daad221b506b43dedadf03f3857d8a563f90"} Jul 06 01:36:51.894754 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:51.894731 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:52.043958 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.043932 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-bundle\") pod \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " Jul 06 01:36:52.044082 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.044022 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg2w9\" (UniqueName: \"kubernetes.io/projected/5279d4cb-7a8c-4b59-a45a-ac16635c2216-kube-api-access-fg2w9\") pod \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " Jul 06 01:36:52.044150 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.044093 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-util\") pod \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\" (UID: \"5279d4cb-7a8c-4b59-a45a-ac16635c2216\") " Jul 06 01:36:52.044556 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.044535 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-bundle" (OuterVolumeSpecName: "bundle") pod "5279d4cb-7a8c-4b59-a45a-ac16635c2216" (UID: "5279d4cb-7a8c-4b59-a45a-ac16635c2216"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:36:52.046358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.046332 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5279d4cb-7a8c-4b59-a45a-ac16635c2216-kube-api-access-fg2w9" (OuterVolumeSpecName: "kube-api-access-fg2w9") pod "5279d4cb-7a8c-4b59-a45a-ac16635c2216" (UID: "5279d4cb-7a8c-4b59-a45a-ac16635c2216"). InnerVolumeSpecName "kube-api-access-fg2w9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:36:52.049455 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.049435 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-util" (OuterVolumeSpecName: "util") pod "5279d4cb-7a8c-4b59-a45a-ac16635c2216" (UID: "5279d4cb-7a8c-4b59-a45a-ac16635c2216"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:36:52.145236 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.145171 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg2w9\" (UniqueName: \"kubernetes.io/projected/5279d4cb-7a8c-4b59-a45a-ac16635c2216-kube-api-access-fg2w9\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:52.145236 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.145201 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-util\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:52.145236 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.145216 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5279d4cb-7a8c-4b59-a45a-ac16635c2216-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:52.761611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.761579 2574 generic.go:356] "Generic (PLEG): container finished" podID="edf02925-4a13-423d-bc70-f477402fd30e" containerID="d9f11aafbf5ac05d1c8cfd183f7ea8407ea17040864d7a7c6e6e657584f2d50e" exitCode=0 Jul 06 01:36:52.762052 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.761663 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" event={"ID":"edf02925-4a13-423d-bc70-f477402fd30e","Type":"ContainerDied","Data":"d9f11aafbf5ac05d1c8cfd183f7ea8407ea17040864d7a7c6e6e657584f2d50e"} Jul 06 01:36:52.763719 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.763696 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" Jul 06 01:36:52.763829 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.763719 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm" event={"ID":"5279d4cb-7a8c-4b59-a45a-ac16635c2216","Type":"ContainerDied","Data":"5617974c16161cddffef297f15b1014efb65423e9ea0221fbf26ab1120be8651"} Jul 06 01:36:52.763829 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.763759 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5617974c16161cddffef297f15b1014efb65423e9ea0221fbf26ab1120be8651" Jul 06 01:36:52.916970 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.916946 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:52.919062 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:52.919042 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:53.053256 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.053166 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-util\") pod \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " Jul 06 01:36:53.053413 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.053275 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-bundle\") pod \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " Jul 06 01:36:53.053413 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.053310 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-util\") pod \"81ed29d4-4135-429a-98cc-c562c1d3fabe\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " Jul 06 01:36:53.053413 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.053352 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsqgp\" (UniqueName: \"kubernetes.io/projected/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-kube-api-access-rsqgp\") pod \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\" (UID: \"35365362-fd3d-4dcd-a422-1aa8acaa9dfb\") " Jul 06 01:36:53.053413 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.053395 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-bundle\") pod \"81ed29d4-4135-429a-98cc-c562c1d3fabe\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " Jul 06 01:36:53.053619 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.053446 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kjvt\" (UniqueName: \"kubernetes.io/projected/81ed29d4-4135-429a-98cc-c562c1d3fabe-kube-api-access-8kjvt\") pod \"81ed29d4-4135-429a-98cc-c562c1d3fabe\" (UID: \"81ed29d4-4135-429a-98cc-c562c1d3fabe\") " Jul 06 01:36:53.053983 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.053955 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-bundle" (OuterVolumeSpecName: "bundle") pod "35365362-fd3d-4dcd-a422-1aa8acaa9dfb" (UID: "35365362-fd3d-4dcd-a422-1aa8acaa9dfb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:36:53.053983 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.053975 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-bundle" (OuterVolumeSpecName: "bundle") pod "81ed29d4-4135-429a-98cc-c562c1d3fabe" (UID: "81ed29d4-4135-429a-98cc-c562c1d3fabe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:36:53.055757 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.055734 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81ed29d4-4135-429a-98cc-c562c1d3fabe-kube-api-access-8kjvt" (OuterVolumeSpecName: "kube-api-access-8kjvt") pod "81ed29d4-4135-429a-98cc-c562c1d3fabe" (UID: "81ed29d4-4135-429a-98cc-c562c1d3fabe"). InnerVolumeSpecName "kube-api-access-8kjvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:36:53.055870 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.055761 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-kube-api-access-rsqgp" (OuterVolumeSpecName: "kube-api-access-rsqgp") pod "35365362-fd3d-4dcd-a422-1aa8acaa9dfb" (UID: "35365362-fd3d-4dcd-a422-1aa8acaa9dfb"). InnerVolumeSpecName "kube-api-access-rsqgp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:36:53.061693 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.061669 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-util" (OuterVolumeSpecName: "util") pod "81ed29d4-4135-429a-98cc-c562c1d3fabe" (UID: "81ed29d4-4135-429a-98cc-c562c1d3fabe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:36:53.062121 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.062102 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-util" (OuterVolumeSpecName: "util") pod "35365362-fd3d-4dcd-a422-1aa8acaa9dfb" (UID: "35365362-fd3d-4dcd-a422-1aa8acaa9dfb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:36:53.154913 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.154853 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kjvt\" (UniqueName: \"kubernetes.io/projected/81ed29d4-4135-429a-98cc-c562c1d3fabe-kube-api-access-8kjvt\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:53.154913 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.154911 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-util\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:53.154913 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.154921 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:53.154913 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.154929 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-util\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:53.155135 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.154937 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsqgp\" (UniqueName: \"kubernetes.io/projected/35365362-fd3d-4dcd-a422-1aa8acaa9dfb-kube-api-access-rsqgp\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:53.155135 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.154947 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81ed29d4-4135-429a-98cc-c562c1d3fabe-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:53.770437 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.770411 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" Jul 06 01:36:53.770831 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.770439 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh" event={"ID":"81ed29d4-4135-429a-98cc-c562c1d3fabe","Type":"ContainerDied","Data":"c533664b578451845ec5c3922f097ea58e81cd8152cc775fc69bab9d66bad776"} Jul 06 01:36:53.770831 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.770488 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c533664b578451845ec5c3922f097ea58e81cd8152cc775fc69bab9d66bad776" Jul 06 01:36:53.772336 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.772317 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" Jul 06 01:36:53.772434 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.772354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw" event={"ID":"35365362-fd3d-4dcd-a422-1aa8acaa9dfb","Type":"ContainerDied","Data":"fc7c89c810d141dde279285bd3d3a31bd16ab7221c65a39fe44a44f3bd194b7e"} Jul 06 01:36:53.772434 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.772384 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc7c89c810d141dde279285bd3d3a31bd16ab7221c65a39fe44a44f3bd194b7e" Jul 06 01:36:53.904330 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.904307 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:53.964073 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.964045 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-util\") pod \"edf02925-4a13-423d-bc70-f477402fd30e\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " Jul 06 01:36:53.964253 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.964127 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg7nc\" (UniqueName: \"kubernetes.io/projected/edf02925-4a13-423d-bc70-f477402fd30e-kube-api-access-dg7nc\") pod \"edf02925-4a13-423d-bc70-f477402fd30e\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " Jul 06 01:36:53.964253 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.964146 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-bundle\") pod \"edf02925-4a13-423d-bc70-f477402fd30e\" (UID: \"edf02925-4a13-423d-bc70-f477402fd30e\") " Jul 06 01:36:53.964520 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.964500 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-bundle" (OuterVolumeSpecName: "bundle") pod "edf02925-4a13-423d-bc70-f477402fd30e" (UID: "edf02925-4a13-423d-bc70-f477402fd30e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:36:53.966495 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.966465 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edf02925-4a13-423d-bc70-f477402fd30e-kube-api-access-dg7nc" (OuterVolumeSpecName: "kube-api-access-dg7nc") pod "edf02925-4a13-423d-bc70-f477402fd30e" (UID: "edf02925-4a13-423d-bc70-f477402fd30e"). InnerVolumeSpecName "kube-api-access-dg7nc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:36:53.970279 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:53.970249 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-util" (OuterVolumeSpecName: "util") pod "edf02925-4a13-423d-bc70-f477402fd30e" (UID: "edf02925-4a13-423d-bc70-f477402fd30e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:36:54.065194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:54.065129 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg7nc\" (UniqueName: \"kubernetes.io/projected/edf02925-4a13-423d-bc70-f477402fd30e-kube-api-access-dg7nc\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:54.065194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:54.065152 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:54.065194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:54.065161 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edf02925-4a13-423d-bc70-f477402fd30e-util\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:36:54.779660 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:54.779572 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" event={"ID":"edf02925-4a13-423d-bc70-f477402fd30e","Type":"ContainerDied","Data":"083248f0f0d8cfa1c8cf7c0174de5e40be428f6f3656eb25cf1a20a7a3ce8549"} Jul 06 01:36:54.779660 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:54.779590 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg" Jul 06 01:36:54.779660 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:54.779621 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="083248f0f0d8cfa1c8cf7c0174de5e40be428f6f3656eb25cf1a20a7a3ce8549" Jul 06 01:36:59.444490 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:59.444455 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:59.444959 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:59.444518 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:59.449717 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:59.449694 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:59.808468 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:59.808391 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d879dd9fb-xf54x" Jul 06 01:36:59.860544 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:36:59.860510 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fc5b7db48-4l4m2"] Jul 06 01:37:08.242041 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242008 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5"] Jul 06 01:37:08.242447 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242433 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerName="util" Jul 06 01:37:08.242502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242448 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerName="util" Jul 06 01:37:08.242502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242457 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerName="extract" Jul 06 01:37:08.242502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242462 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerName="extract" Jul 06 01:37:08.242502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242470 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edf02925-4a13-423d-bc70-f477402fd30e" containerName="pull" Jul 06 01:37:08.242502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242479 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="edf02925-4a13-423d-bc70-f477402fd30e" containerName="pull" Jul 06 01:37:08.242502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242489 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerName="util" Jul 06 01:37:08.242502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242494 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerName="util" Jul 06 01:37:08.242502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242501 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerName="pull" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242506 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerName="pull" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242513 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242517 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242529 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerName="pull" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242534 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerName="pull" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242541 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242549 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242563 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerName="util" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242568 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerName="util" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242573 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edf02925-4a13-423d-bc70-f477402fd30e" containerName="util" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242577 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="edf02925-4a13-423d-bc70-f477402fd30e" containerName="util" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242588 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerName="pull" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242592 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerName="pull" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242599 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edf02925-4a13-423d-bc70-f477402fd30e" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242603 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="edf02925-4a13-423d-bc70-f477402fd30e" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242690 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5279d4cb-7a8c-4b59-a45a-ac16635c2216" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242702 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="35365362-fd3d-4dcd-a422-1aa8acaa9dfb" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242715 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="81ed29d4-4135-429a-98cc-c562c1d3fabe" containerName="extract" Jul 06 01:37:08.242728 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.242723 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="edf02925-4a13-423d-bc70-f477402fd30e" containerName="extract" Jul 06 01:37:08.245912 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.245898 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" Jul 06 01:37:08.248204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.248183 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 01:37:08.248344 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.248244 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 01:37:08.248801 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.248787 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 01:37:08.248887 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.248798 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lcsvq\"" Jul 06 01:37:08.257822 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.257802 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5"] Jul 06 01:37:08.389231 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.389203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgv2r\" (UniqueName: \"kubernetes.io/projected/e12b1f19-8e4c-4507-a8f7-d485b96ebb63-kube-api-access-sgv2r\") pod \"dns-operator-controller-manager-864dd78c4c-4nbj5\" (UID: \"e12b1f19-8e4c-4507-a8f7-d485b96ebb63\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" Jul 06 01:37:08.490650 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.490617 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgv2r\" (UniqueName: \"kubernetes.io/projected/e12b1f19-8e4c-4507-a8f7-d485b96ebb63-kube-api-access-sgv2r\") pod \"dns-operator-controller-manager-864dd78c4c-4nbj5\" (UID: \"e12b1f19-8e4c-4507-a8f7-d485b96ebb63\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" Jul 06 01:37:08.504060 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.504002 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgv2r\" (UniqueName: \"kubernetes.io/projected/e12b1f19-8e4c-4507-a8f7-d485b96ebb63-kube-api-access-sgv2r\") pod \"dns-operator-controller-manager-864dd78c4c-4nbj5\" (UID: \"e12b1f19-8e4c-4507-a8f7-d485b96ebb63\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" Jul 06 01:37:08.556390 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.556361 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" Jul 06 01:37:08.693317 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.693218 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5"] Jul 06 01:37:08.696104 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:37:08.696066 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12b1f19_8e4c_4507_a8f7_d485b96ebb63.slice/crio-24af328eef273d2c8f2707b93c159cc26dc2f3b83db8c1e2dc649ddefc3babfd WatchSource:0}: Error finding container 24af328eef273d2c8f2707b93c159cc26dc2f3b83db8c1e2dc649ddefc3babfd: Status 404 returned error can't find the container with id 24af328eef273d2c8f2707b93c159cc26dc2f3b83db8c1e2dc649ddefc3babfd Jul 06 01:37:08.858121 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:08.858092 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" event={"ID":"e12b1f19-8e4c-4507-a8f7-d485b96ebb63","Type":"ContainerStarted","Data":"24af328eef273d2c8f2707b93c159cc26dc2f3b83db8c1e2dc649ddefc3babfd"} Jul 06 01:37:10.871238 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:10.871197 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" event={"ID":"e12b1f19-8e4c-4507-a8f7-d485b96ebb63","Type":"ContainerStarted","Data":"5c570ad2404ebb925c7a85e34ed5cb031ce1c94d1677b77cda9aa0c13cbf21d4"} Jul 06 01:37:10.871628 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:10.871373 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" Jul 06 01:37:10.890327 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:10.890280 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" podStartSLOduration=1.119985359 podStartE2EDuration="2.890266851s" totalImagesPullingTime="1.770281492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:37:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:37:10.887756248 +0000 UTC m=+629.023046749" watchObservedRunningTime="2026-07-06 01:37:10.890266851 +0000 UTC m=+629.025557356" Jul 06 01:37:11.316424 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.316354 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg"] Jul 06 01:37:11.320008 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.319993 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" Jul 06 01:37:11.321969 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.321955 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fk8m2\"" Jul 06 01:37:11.330182 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.330163 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg"] Jul 06 01:37:11.417922 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.417894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k76mb\" (UniqueName: \"kubernetes.io/projected/56da7f46-ffea-4848-8458-dec5bf391c6f-kube-api-access-k76mb\") pod \"authorino-operator-75cc8d5cb7-dkvhg\" (UID: \"56da7f46-ffea-4848-8458-dec5bf391c6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" Jul 06 01:37:11.519231 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.519193 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k76mb\" (UniqueName: \"kubernetes.io/projected/56da7f46-ffea-4848-8458-dec5bf391c6f-kube-api-access-k76mb\") pod \"authorino-operator-75cc8d5cb7-dkvhg\" (UID: \"56da7f46-ffea-4848-8458-dec5bf391c6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" Jul 06 01:37:11.527113 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.527087 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k76mb\" (UniqueName: \"kubernetes.io/projected/56da7f46-ffea-4848-8458-dec5bf391c6f-kube-api-access-k76mb\") pod \"authorino-operator-75cc8d5cb7-dkvhg\" (UID: \"56da7f46-ffea-4848-8458-dec5bf391c6f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" Jul 06 01:37:11.630579 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.630556 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" Jul 06 01:37:11.767267 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.767241 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg"] Jul 06 01:37:11.768935 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:37:11.768904 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56da7f46_ffea_4848_8458_dec5bf391c6f.slice/crio-e3a5e79150a94c32f73ae7ab1dbe6e133997916f6ef8160daeeaf9c7f6456986 WatchSource:0}: Error finding container e3a5e79150a94c32f73ae7ab1dbe6e133997916f6ef8160daeeaf9c7f6456986: Status 404 returned error can't find the container with id e3a5e79150a94c32f73ae7ab1dbe6e133997916f6ef8160daeeaf9c7f6456986 Jul 06 01:37:11.877917 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:11.877889 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" event={"ID":"56da7f46-ffea-4848-8458-dec5bf391c6f","Type":"ContainerStarted","Data":"e3a5e79150a94c32f73ae7ab1dbe6e133997916f6ef8160daeeaf9c7f6456986"} Jul 06 01:37:14.899272 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:14.899237 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" event={"ID":"56da7f46-ffea-4848-8458-dec5bf391c6f","Type":"ContainerStarted","Data":"4ce1dcb85a1cdce23e29f84d2a1d65696ef17d7390b0f9a3247d531257db3977"} Jul 06 01:37:14.899671 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:14.899394 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" Jul 06 01:37:14.918802 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:14.918752 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" podStartSLOduration=1.721625779 podStartE2EDuration="3.918736825s" totalImagesPullingTime="2.197111046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:37:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:37:14.916440266 +0000 UTC m=+633.051730769" watchObservedRunningTime="2026-07-06 01:37:14.918736825 +0000 UTC m=+633.054027328" Jul 06 01:37:21.880564 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:21.880536 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nbj5" Jul 06 01:37:24.885959 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:24.885919 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5fc5b7db48-4l4m2" podUID="0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" containerName="console" containerID="cri-o://e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e" gracePeriod=15 Jul 06 01:37:25.129112 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.128071 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:37:25.224208 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224124 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-service-ca\") pod \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " Jul 06 01:37:25.224208 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224160 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-trusted-ca-bundle\") pod \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " Jul 06 01:37:25.224208 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224198 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvjd8\" (UniqueName: \"kubernetes.io/projected/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-kube-api-access-bvjd8\") pod \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " Jul 06 01:37:25.224484 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224224 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-config\") pod \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " Jul 06 01:37:25.224484 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224271 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-oauth-serving-cert\") pod \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " Jul 06 01:37:25.224484 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224302 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-serving-cert\") pod \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " Jul 06 01:37:25.224484 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224366 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-oauth-config\") pod \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\" (UID: \"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90\") " Jul 06 01:37:25.224673 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224568 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" (UID: "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:37:25.224673 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224640 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" (UID: "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:37:25.224775 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224715 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-config" (OuterVolumeSpecName: "console-config") pod "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" (UID: "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:37:25.224827 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224773 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-service-ca\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:25.224827 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224772 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" (UID: "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:37:25.224827 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.224793 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-trusted-ca-bundle\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:25.226634 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.226610 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" (UID: "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:37:25.226718 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.226661 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-kube-api-access-bvjd8" (OuterVolumeSpecName: "kube-api-access-bvjd8") pod "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" (UID: "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90"). InnerVolumeSpecName "kube-api-access-bvjd8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:37:25.226718 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.226699 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" (UID: "0e26b553-4e78-4bb8-b5bd-a7e9180aaa90"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:37:25.325759 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.325726 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvjd8\" (UniqueName: \"kubernetes.io/projected/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-kube-api-access-bvjd8\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:25.325759 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.325754 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-config\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:25.325759 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.325766 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-oauth-serving-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:25.326014 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.325780 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-serving-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:25.326014 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.325792 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90-console-oauth-config\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:25.856795 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.856760 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225"] Jul 06 01:37:25.857198 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.857182 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" containerName="console" Jul 06 01:37:25.857261 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.857201 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" containerName="console" Jul 06 01:37:25.857299 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.857294 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" containerName="console" Jul 06 01:37:25.861302 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.861288 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:25.863478 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.863461 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xkfc6\"" Jul 06 01:37:25.872811 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.872785 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225"] Jul 06 01:37:25.907210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.907182 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-dkvhg" Jul 06 01:37:25.932064 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.932031 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/293b0865-6e27-4463-8898-e850638ab733-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-sg225\" (UID: \"293b0865-6e27-4463-8898-e850638ab733\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:25.932209 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.932079 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6ksg\" (UniqueName: \"kubernetes.io/projected/293b0865-6e27-4463-8898-e850638ab733-kube-api-access-c6ksg\") pod \"kuadrant-operator-controller-manager-55467d9f7-sg225\" (UID: \"293b0865-6e27-4463-8898-e850638ab733\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:25.960640 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.960609 2574 generic.go:356] "Generic (PLEG): container finished" podID="0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" containerID="e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e" exitCode=0 Jul 06 01:37:25.960806 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.960656 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fc5b7db48-4l4m2" event={"ID":"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90","Type":"ContainerDied","Data":"e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e"} Jul 06 01:37:25.960806 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.960667 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fc5b7db48-4l4m2" Jul 06 01:37:25.960806 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.960700 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fc5b7db48-4l4m2" event={"ID":"0e26b553-4e78-4bb8-b5bd-a7e9180aaa90","Type":"ContainerDied","Data":"81a1c5cd360a4e48d3589e2dad7662e3d7b9e5950850193d0ba843012a3fcee1"} Jul 06 01:37:25.960806 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.960719 2574 scope.go:117] "RemoveContainer" containerID="e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e" Jul 06 01:37:25.968068 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.968045 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225"] Jul 06 01:37:25.968417 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:37:25.968397 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-c6ksg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" podUID="293b0865-6e27-4463-8898-e850638ab733" Jul 06 01:37:25.972329 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.972312 2574 scope.go:117] "RemoveContainer" containerID="e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e" Jul 06 01:37:25.972648 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:37:25.972614 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e\": container with ID starting with e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e not found: ID does not exist" containerID="e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e" Jul 06 01:37:25.972648 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.972643 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e"} err="failed to get container status \"e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e\": rpc error: code = NotFound desc = could not find container \"e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e\": container with ID starting with e8ffafd40326f9850b08818814ae004f40c8ebc25c4a55cd777789ce8fbc307e not found: ID does not exist" Jul 06 01:37:25.992089 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.992067 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fc5b7db48-4l4m2"] Jul 06 01:37:26.000016 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:25.999999 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fc5b7db48-4l4m2"] Jul 06 01:37:26.033156 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.033068 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/293b0865-6e27-4463-8898-e850638ab733-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-sg225\" (UID: \"293b0865-6e27-4463-8898-e850638ab733\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:26.033156 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.033114 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6ksg\" (UniqueName: \"kubernetes.io/projected/293b0865-6e27-4463-8898-e850638ab733-kube-api-access-c6ksg\") pod \"kuadrant-operator-controller-manager-55467d9f7-sg225\" (UID: \"293b0865-6e27-4463-8898-e850638ab733\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:26.033635 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.033616 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/293b0865-6e27-4463-8898-e850638ab733-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-sg225\" (UID: \"293b0865-6e27-4463-8898-e850638ab733\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:26.052940 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.052919 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6ksg\" (UniqueName: \"kubernetes.io/projected/293b0865-6e27-4463-8898-e850638ab733-kube-api-access-c6ksg\") pod \"kuadrant-operator-controller-manager-55467d9f7-sg225\" (UID: \"293b0865-6e27-4463-8898-e850638ab733\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:26.087689 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.087657 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225"] Jul 06 01:37:26.092451 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.092428 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225"] Jul 06 01:37:26.127209 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.127057 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn"] Jul 06 01:37:26.132570 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.132551 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:26.145124 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.145101 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn"] Jul 06 01:37:26.234801 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.234761 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8be7bec-656e-4499-aa24-3b25bc562838-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxczn\" (UID: \"c8be7bec-656e-4499-aa24-3b25bc562838\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:26.235006 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.234943 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmlqz\" (UniqueName: \"kubernetes.io/projected/c8be7bec-656e-4499-aa24-3b25bc562838-kube-api-access-cmlqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxczn\" (UID: \"c8be7bec-656e-4499-aa24-3b25bc562838\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:26.335650 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.335605 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8be7bec-656e-4499-aa24-3b25bc562838-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxczn\" (UID: \"c8be7bec-656e-4499-aa24-3b25bc562838\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:26.335825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.335706 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmlqz\" (UniqueName: \"kubernetes.io/projected/c8be7bec-656e-4499-aa24-3b25bc562838-kube-api-access-cmlqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxczn\" (UID: \"c8be7bec-656e-4499-aa24-3b25bc562838\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:26.336039 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.336018 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8be7bec-656e-4499-aa24-3b25bc562838-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxczn\" (UID: \"c8be7bec-656e-4499-aa24-3b25bc562838\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:26.357884 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.357822 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmlqz\" (UniqueName: \"kubernetes.io/projected/c8be7bec-656e-4499-aa24-3b25bc562838-kube-api-access-cmlqz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fxczn\" (UID: \"c8be7bec-656e-4499-aa24-3b25bc562838\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:26.381642 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.381577 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e26b553-4e78-4bb8-b5bd-a7e9180aaa90" path="/var/lib/kubelet/pods/0e26b553-4e78-4bb8-b5bd-a7e9180aaa90/volumes" Jul 06 01:37:26.444555 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.444526 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:26.579013 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.578984 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn"] Jul 06 01:37:26.581218 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:37:26.581147 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8be7bec_656e_4499_aa24_3b25bc562838.slice/crio-1bda6c025fc6f2b10588f2b839df0d8392f228dfa21f425b04ec5198d0e028e8 WatchSource:0}: Error finding container 1bda6c025fc6f2b10588f2b839df0d8392f228dfa21f425b04ec5198d0e028e8: Status 404 returned error can't find the container with id 1bda6c025fc6f2b10588f2b839df0d8392f228dfa21f425b04ec5198d0e028e8 Jul 06 01:37:26.967895 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.967843 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" event={"ID":"c8be7bec-656e-4499-aa24-3b25bc562838","Type":"ContainerStarted","Data":"1bda6c025fc6f2b10588f2b839df0d8392f228dfa21f425b04ec5198d0e028e8"} Jul 06 01:37:26.968700 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.968686 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:26.977095 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:26.977077 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:26.978783 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:37:26.978759 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-sg225\" is forbidden: User \"system:node:ip-10-0-133-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-193.ec2.internal' and this object" podUID="293b0865-6e27-4463-8898-e850638ab733" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:27.041182 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:27.041155 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6ksg\" (UniqueName: \"kubernetes.io/projected/293b0865-6e27-4463-8898-e850638ab733-kube-api-access-c6ksg\") pod \"293b0865-6e27-4463-8898-e850638ab733\" (UID: \"293b0865-6e27-4463-8898-e850638ab733\") " Jul 06 01:37:27.041336 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:27.041267 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/293b0865-6e27-4463-8898-e850638ab733-extensions-socket-volume\") pod \"293b0865-6e27-4463-8898-e850638ab733\" (UID: \"293b0865-6e27-4463-8898-e850638ab733\") " Jul 06 01:37:27.041535 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:27.041513 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b0865-6e27-4463-8898-e850638ab733-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "293b0865-6e27-4463-8898-e850638ab733" (UID: "293b0865-6e27-4463-8898-e850638ab733"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:37:27.043294 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:27.043270 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293b0865-6e27-4463-8898-e850638ab733-kube-api-access-c6ksg" (OuterVolumeSpecName: "kube-api-access-c6ksg") pod "293b0865-6e27-4463-8898-e850638ab733" (UID: "293b0865-6e27-4463-8898-e850638ab733"). InnerVolumeSpecName "kube-api-access-c6ksg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:37:27.142580 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:27.142547 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/293b0865-6e27-4463-8898-e850638ab733-extensions-socket-volume\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:27.142580 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:27.142577 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6ksg\" (UniqueName: \"kubernetes.io/projected/293b0865-6e27-4463-8898-e850638ab733-kube-api-access-c6ksg\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:27.975020 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:27.974987 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:27.976964 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:37:27.976931 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-sg225\" is forbidden: User \"system:node:ip-10-0-133-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-193.ec2.internal' and this object" podUID="293b0865-6e27-4463-8898-e850638ab733" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:27.983106 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:37:27.983076 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-sg225\" is forbidden: User \"system:node:ip-10-0-133-193.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-193.ec2.internal' and this object" podUID="293b0865-6e27-4463-8898-e850638ab733" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sg225" Jul 06 01:37:28.385062 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:28.385032 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="293b0865-6e27-4463-8898-e850638ab733" path="/var/lib/kubelet/pods/293b0865-6e27-4463-8898-e850638ab733/volumes" Jul 06 01:37:30.996498 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:30.996457 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" event={"ID":"c8be7bec-656e-4499-aa24-3b25bc562838","Type":"ContainerStarted","Data":"a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc"} Jul 06 01:37:30.996948 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:30.996540 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:31.015468 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:31.015421 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" podStartSLOduration=0.714063323 podStartE2EDuration="5.015406237s" totalImagesPullingTime="4.301342914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:37:31.013838468 +0000 UTC m=+649.149128971" watchObservedRunningTime="2026-07-06 01:37:31.015406237 +0000 UTC m=+649.150696742" Jul 06 01:37:38.269436 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.269385 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn"] Jul 06 01:37:38.269845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.269616 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" podUID="c8be7bec-656e-4499-aa24-3b25bc562838" containerName="manager" containerID="cri-o://a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc" gracePeriod=10 Jul 06 01:37:38.271364 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.271336 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:38.525293 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.525238 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:38.651230 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.651195 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmlqz\" (UniqueName: \"kubernetes.io/projected/c8be7bec-656e-4499-aa24-3b25bc562838-kube-api-access-cmlqz\") pod \"c8be7bec-656e-4499-aa24-3b25bc562838\" (UID: \"c8be7bec-656e-4499-aa24-3b25bc562838\") " Jul 06 01:37:38.651399 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.651298 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8be7bec-656e-4499-aa24-3b25bc562838-extensions-socket-volume\") pod \"c8be7bec-656e-4499-aa24-3b25bc562838\" (UID: \"c8be7bec-656e-4499-aa24-3b25bc562838\") " Jul 06 01:37:38.651765 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.651743 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8be7bec-656e-4499-aa24-3b25bc562838-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8be7bec-656e-4499-aa24-3b25bc562838" (UID: "c8be7bec-656e-4499-aa24-3b25bc562838"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:37:38.653441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.653416 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8be7bec-656e-4499-aa24-3b25bc562838-kube-api-access-cmlqz" (OuterVolumeSpecName: "kube-api-access-cmlqz") pod "c8be7bec-656e-4499-aa24-3b25bc562838" (UID: "c8be7bec-656e-4499-aa24-3b25bc562838"). InnerVolumeSpecName "kube-api-access-cmlqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:37:38.752853 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.752826 2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8be7bec-656e-4499-aa24-3b25bc562838-extensions-socket-volume\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:38.752976 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:38.752883 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmlqz\" (UniqueName: \"kubernetes.io/projected/c8be7bec-656e-4499-aa24-3b25bc562838-kube-api-access-cmlqz\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:37:39.038311 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.038282 2574 generic.go:356] "Generic (PLEG): container finished" podID="c8be7bec-656e-4499-aa24-3b25bc562838" containerID="a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc" exitCode=0 Jul 06 01:37:39.038489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.038318 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" event={"ID":"c8be7bec-656e-4499-aa24-3b25bc562838","Type":"ContainerDied","Data":"a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc"} Jul 06 01:37:39.038489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.038327 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" Jul 06 01:37:39.038489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.038354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn" event={"ID":"c8be7bec-656e-4499-aa24-3b25bc562838","Type":"ContainerDied","Data":"1bda6c025fc6f2b10588f2b839df0d8392f228dfa21f425b04ec5198d0e028e8"} Jul 06 01:37:39.038489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.038374 2574 scope.go:117] "RemoveContainer" containerID="a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc" Jul 06 01:37:39.048505 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.048490 2574 scope.go:117] "RemoveContainer" containerID="a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc" Jul 06 01:37:39.048744 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:37:39.048724 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc\": container with ID starting with a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc not found: ID does not exist" containerID="a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc" Jul 06 01:37:39.048794 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.048752 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc"} err="failed to get container status \"a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc\": rpc error: code = NotFound desc = could not find container \"a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc\": container with ID starting with a36826d012a632152b8ba1c3eafd00bf94215ec9b54cda8d600b0a8cd8c4b3fc not found: ID does not exist" Jul 06 01:37:39.063877 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.061620 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn"] Jul 06 01:37:39.068564 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:39.068543 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fxczn"] Jul 06 01:37:40.381885 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:40.381841 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8be7bec-656e-4499-aa24-3b25bc562838" path="/var/lib/kubelet/pods/c8be7bec-656e-4499-aa24-3b25bc562838/volumes" Jul 06 01:37:54.745027 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.744994 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7"] Jul 06 01:37:54.745431 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.745408 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8be7bec-656e-4499-aa24-3b25bc562838" containerName="manager" Jul 06 01:37:54.745431 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.745427 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8be7bec-656e-4499-aa24-3b25bc562838" containerName="manager" Jul 06 01:37:54.745595 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.745496 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8be7bec-656e-4499-aa24-3b25bc562838" containerName="manager" Jul 06 01:37:54.750999 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.750983 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.753049 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.753029 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wc4zq\"" Jul 06 01:37:54.759284 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.759252 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7"] Jul 06 01:37:54.891370 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891332 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.891370 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891370 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwktw\" (UniqueName: \"kubernetes.io/projected/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-kube-api-access-lwktw\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.891571 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891394 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.891571 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891451 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.891571 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891472 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.891571 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891520 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.891571 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891549 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.892055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.892055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.891613 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992259 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992223 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992259 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992485 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992309 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992485 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwktw\" (UniqueName: \"kubernetes.io/projected/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-kube-api-access-lwktw\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992485 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992485 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992385 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992485 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992400 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992742 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992608 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992742 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992676 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992850 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992796 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.992850 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.992827 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.993060 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.993041 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.993125 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.993095 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.993205 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.993184 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.995020 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.995000 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.995127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.995030 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.999408 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.999386 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:54.999497 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:54.999412 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwktw\" (UniqueName: \"kubernetes.io/projected/abde296b-bf0d-45e4-8db4-d79ee2e60e0f-kube-api-access-lwktw\") pod \"maas-default-gateway-openshift-default-56654dcf7-z8wf7\" (UID: \"abde296b-bf0d-45e4-8db4-d79ee2e60e0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:55.062555 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:55.062533 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:55.193440 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:55.193418 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7"] Jul 06 01:37:55.195190 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:37:55.195166 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabde296b_bf0d_45e4_8db4_d79ee2e60e0f.slice/crio-6ba3728c62c0b42ffa5414eebca71863061bc415132e0517548ed87f1f549187 WatchSource:0}: Error finding container 6ba3728c62c0b42ffa5414eebca71863061bc415132e0517548ed87f1f549187: Status 404 returned error can't find the container with id 6ba3728c62c0b42ffa5414eebca71863061bc415132e0517548ed87f1f549187 Jul 06 01:37:55.197216 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:55.197183 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:37:55.197301 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:55.197245 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:37:55.197301 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:55.197273 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:37:56.132803 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:56.132768 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" event={"ID":"abde296b-bf0d-45e4-8db4-d79ee2e60e0f","Type":"ContainerStarted","Data":"08a8f87b98e88bad8d299a50077a89114d5d20b763f72bbf66deee328d91fa0c"} Jul 06 01:37:56.133279 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:56.132828 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" event={"ID":"abde296b-bf0d-45e4-8db4-d79ee2e60e0f","Type":"ContainerStarted","Data":"6ba3728c62c0b42ffa5414eebca71863061bc415132e0517548ed87f1f549187"} Jul 06 01:37:56.151422 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:56.151376 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" podStartSLOduration=2.1513614 podStartE2EDuration="2.1513614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:37:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:37:56.149311808 +0000 UTC m=+674.284602311" watchObservedRunningTime="2026-07-06 01:37:56.1513614 +0000 UTC m=+674.286651904" Jul 06 01:37:57.062651 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.062623 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:57.067324 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.067303 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:57.138358 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.138333 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:57.139130 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.139112 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:37:57.243653 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.243617 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6wgwq"] Jul 06 01:37:57.247975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.247958 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:37:57.250503 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.250482 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cgz9t\"" Jul 06 01:37:57.250639 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.250482 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 01:37:57.252538 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.252513 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6wgwq"] Jul 06 01:37:57.335088 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.335052 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6wgwq"] Jul 06 01:37:57.419066 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.419030 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5ldz\" (UniqueName: \"kubernetes.io/projected/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-kube-api-access-w5ldz\") pod \"limitador-limitador-64c94fd64f-6wgwq\" (UID: \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:37:57.419224 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.419099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-config-file\") pod \"limitador-limitador-64c94fd64f-6wgwq\" (UID: \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:37:57.520315 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.520278 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-config-file\") pod \"limitador-limitador-64c94fd64f-6wgwq\" (UID: \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:37:57.520502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.520378 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5ldz\" (UniqueName: \"kubernetes.io/projected/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-kube-api-access-w5ldz\") pod \"limitador-limitador-64c94fd64f-6wgwq\" (UID: \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:37:57.521007 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.520986 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-config-file\") pod \"limitador-limitador-64c94fd64f-6wgwq\" (UID: \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:37:57.527571 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.527545 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5ldz\" (UniqueName: \"kubernetes.io/projected/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-kube-api-access-w5ldz\") pod \"limitador-limitador-64c94fd64f-6wgwq\" (UID: \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:37:57.562172 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.562148 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:37:57.694982 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:57.694946 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6wgwq"] Jul 06 01:37:57.696755 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:37:57.696719 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf47c408_f7ea_41a7_9643_6a6e54fd4e6d.slice/crio-4770a9d06038cd292bf882cd419a235b400e9640ab74f9999e5b87a53d7394ff WatchSource:0}: Error finding container 4770a9d06038cd292bf882cd419a235b400e9640ab74f9999e5b87a53d7394ff: Status 404 returned error can't find the container with id 4770a9d06038cd292bf882cd419a235b400e9640ab74f9999e5b87a53d7394ff Jul 06 01:37:58.145491 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:37:58.145460 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" event={"ID":"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d","Type":"ContainerStarted","Data":"4770a9d06038cd292bf882cd419a235b400e9640ab74f9999e5b87a53d7394ff"} Jul 06 01:38:01.165911 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:01.165853 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" event={"ID":"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d","Type":"ContainerStarted","Data":"5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3"} Jul 06 01:38:01.166299 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:01.165992 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:38:01.182834 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:01.182785 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" podStartSLOduration=1.260714007 podStartE2EDuration="4.182772263s" totalImagesPullingTime="2.922058256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:38:01.180315694 +0000 UTC m=+679.315606199" watchObservedRunningTime="2026-07-06 01:38:01.182772263 +0000 UTC m=+679.318062767" Jul 06 01:38:12.172383 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:12.172356 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:38:12.424091 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:12.424007 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6wgwq"] Jul 06 01:38:12.424285 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:12.424177 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" podUID="cf47c408-f7ea-41a7-9643-6a6e54fd4e6d" containerName="limitador" containerID="cri-o://5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3" gracePeriod=30 Jul 06 01:38:12.972911 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:12.972886 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:38:13.060067 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.059988 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5ldz\" (UniqueName: \"kubernetes.io/projected/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-kube-api-access-w5ldz\") pod \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\" (UID: \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\") " Jul 06 01:38:13.060067 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.060046 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-config-file\") pod \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\" (UID: \"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d\") " Jul 06 01:38:13.060373 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.060355 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-config-file" (OuterVolumeSpecName: "config-file") pod "cf47c408-f7ea-41a7-9643-6a6e54fd4e6d" (UID: "cf47c408-f7ea-41a7-9643-6a6e54fd4e6d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:38:13.062210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.062189 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-kube-api-access-w5ldz" (OuterVolumeSpecName: "kube-api-access-w5ldz") pod "cf47c408-f7ea-41a7-9643-6a6e54fd4e6d" (UID: "cf47c408-f7ea-41a7-9643-6a6e54fd4e6d"). InnerVolumeSpecName "kube-api-access-w5ldz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:38:13.161643 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.161607 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5ldz\" (UniqueName: \"kubernetes.io/projected/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-kube-api-access-w5ldz\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:38:13.161643 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.161640 2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d-config-file\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:38:13.230679 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.230646 2574 generic.go:356] "Generic (PLEG): container finished" podID="cf47c408-f7ea-41a7-9643-6a6e54fd4e6d" containerID="5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3" exitCode=0 Jul 06 01:38:13.231129 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.230693 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" Jul 06 01:38:13.231129 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.230728 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" event={"ID":"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d","Type":"ContainerDied","Data":"5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3"} Jul 06 01:38:13.231129 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.230779 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-6wgwq" event={"ID":"cf47c408-f7ea-41a7-9643-6a6e54fd4e6d","Type":"ContainerDied","Data":"4770a9d06038cd292bf882cd419a235b400e9640ab74f9999e5b87a53d7394ff"} Jul 06 01:38:13.231129 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.230800 2574 scope.go:117] "RemoveContainer" containerID="5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3" Jul 06 01:38:13.240224 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.240207 2574 scope.go:117] "RemoveContainer" containerID="5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3" Jul 06 01:38:13.240458 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:38:13.240443 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3\": container with ID starting with 5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3 not found: ID does not exist" containerID="5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3" Jul 06 01:38:13.240519 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.240464 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3"} err="failed to get container status \"5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3\": rpc error: code = NotFound desc = could not find container \"5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3\": container with ID starting with 5d3449f42b4b721d7f64a91b402aaf4c66ef4df36e7c8835b9f7c89830769aa3 not found: ID does not exist" Jul 06 01:38:13.250983 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.250962 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6wgwq"] Jul 06 01:38:13.253059 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:13.253041 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-6wgwq"] Jul 06 01:38:14.382181 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:14.382149 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf47c408-f7ea-41a7-9643-6a6e54fd4e6d" path="/var/lib/kubelet/pods/cf47c408-f7ea-41a7-9643-6a6e54fd4e6d/volumes" Jul 06 01:38:18.950508 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:18.950476 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-8m2q9"] Jul 06 01:38:18.951070 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:18.951053 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf47c408-f7ea-41a7-9643-6a6e54fd4e6d" containerName="limitador" Jul 06 01:38:18.951120 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:18.951073 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf47c408-f7ea-41a7-9643-6a6e54fd4e6d" containerName="limitador" Jul 06 01:38:18.951201 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:18.951189 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf47c408-f7ea-41a7-9643-6a6e54fd4e6d" containerName="limitador" Jul 06 01:38:18.955908 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:18.955888 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:18.957869 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:18.957840 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 01:38:18.957976 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:18.957881 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qzktr\"" Jul 06 01:38:18.965118 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:18.965096 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-8m2q9"] Jul 06 01:38:19.015497 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:19.015468 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7vnb\" (UniqueName: \"kubernetes.io/projected/433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9-kube-api-access-p7vnb\") pod \"postgres-69fb4594cf-8m2q9\" (UID: \"433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9\") " pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:19.015655 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:19.015591 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9-data\") pod \"postgres-69fb4594cf-8m2q9\" (UID: \"433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9\") " pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:19.116412 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:19.116376 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9-data\") pod \"postgres-69fb4594cf-8m2q9\" (UID: \"433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9\") " pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:19.116613 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:19.116423 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7vnb\" (UniqueName: \"kubernetes.io/projected/433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9-kube-api-access-p7vnb\") pod \"postgres-69fb4594cf-8m2q9\" (UID: \"433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9\") " pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:19.117175 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:19.117146 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9-data\") pod \"postgres-69fb4594cf-8m2q9\" (UID: \"433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9\") " pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:19.124636 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:19.124614 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7vnb\" (UniqueName: \"kubernetes.io/projected/433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9-kube-api-access-p7vnb\") pod \"postgres-69fb4594cf-8m2q9\" (UID: \"433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9\") " pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:19.268140 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:19.268055 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:19.395939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:19.395915 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-8m2q9"] Jul 06 01:38:19.398000 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:38:19.397974 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433ef9c0_615f_4cd4_ba2c_91b3bdefc0c9.slice/crio-b25ba940a20de81c6cbb8899fcbe1f60a3837f13f6d945b51a05a525693b2da0 WatchSource:0}: Error finding container b25ba940a20de81c6cbb8899fcbe1f60a3837f13f6d945b51a05a525693b2da0: Status 404 returned error can't find the container with id b25ba940a20de81c6cbb8899fcbe1f60a3837f13f6d945b51a05a525693b2da0 Jul 06 01:38:20.278235 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:20.278194 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-8m2q9" event={"ID":"433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9","Type":"ContainerStarted","Data":"b25ba940a20de81c6cbb8899fcbe1f60a3837f13f6d945b51a05a525693b2da0"} Jul 06 01:38:24.304175 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:24.304145 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-8m2q9" event={"ID":"433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9","Type":"ContainerStarted","Data":"bb8755119e48378ae072392ca9976850f7c93512a03da5eee0d4925ee3e192b6"} Jul 06 01:38:24.304555 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:24.304271 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:24.317611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:24.317559 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-8m2q9" podStartSLOduration=1.601493678 podStartE2EDuration="6.317545497s" totalImagesPullingTime="4.716051819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:38:24.316836229 +0000 UTC m=+702.452126733" watchObservedRunningTime="2026-07-06 01:38:24.317545497 +0000 UTC m=+702.452836000" Jul 06 01:38:30.340102 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:30.340069 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-8m2q9" Jul 06 01:38:38.161392 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.161315 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c8865c55d-z45zf"] Jul 06 01:38:38.170567 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.170540 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:38.172843 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.172818 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 01:38:38.172998 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.172848 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 01:38:38.172998 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.172848 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dr8hv\"" Jul 06 01:38:38.173992 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.173970 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8865c55d-z45zf"] Jul 06 01:38:38.293021 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.292985 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert\") pod \"maas-controller-6c8865c55d-z45zf\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:38.293191 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.293041 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxcqv\" (UniqueName: \"kubernetes.io/projected/3d382abe-586d-4bab-834a-3527cb48f041-kube-api-access-vxcqv\") pod \"maas-controller-6c8865c55d-z45zf\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:38.349393 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.349360 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jnvtz"] Jul 06 01:38:38.352626 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.352611 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:38.360760 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.360741 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jnvtz"] Jul 06 01:38:38.394367 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.394335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert\") pod \"maas-controller-6c8865c55d-z45zf\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:38.394518 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.394397 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxcqv\" (UniqueName: \"kubernetes.io/projected/3d382abe-586d-4bab-834a-3527cb48f041-kube-api-access-vxcqv\") pod \"maas-controller-6c8865c55d-z45zf\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:38.394518 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:38:38.394487 2574 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 01:38:38.394614 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:38:38.394549 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert podName:3d382abe-586d-4bab-834a-3527cb48f041 nodeName:}" failed. No retries permitted until 2026-07-06 01:38:38.894530931 +0000 UTC m=+717.029821413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert") pod "maas-controller-6c8865c55d-z45zf" (UID: "3d382abe-586d-4bab-834a-3527cb48f041") : secret "maas-controller-webhook-cert" not found Jul 06 01:38:38.404401 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.404375 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxcqv\" (UniqueName: \"kubernetes.io/projected/3d382abe-586d-4bab-834a-3527cb48f041-kube-api-access-vxcqv\") pod \"maas-controller-6c8865c55d-z45zf\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:38.495443 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.495367 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fab7dcb-4329-4026-800c-2ea78618fcd7-webhook-cert\") pod \"maas-controller-6d6f449ff5-jnvtz\" (UID: \"7fab7dcb-4329-4026-800c-2ea78618fcd7\") " pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:38.495593 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.495570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gl8g\" (UniqueName: \"kubernetes.io/projected/7fab7dcb-4329-4026-800c-2ea78618fcd7-kube-api-access-5gl8g\") pod \"maas-controller-6d6f449ff5-jnvtz\" (UID: \"7fab7dcb-4329-4026-800c-2ea78618fcd7\") " pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:38.596735 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.596698 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fab7dcb-4329-4026-800c-2ea78618fcd7-webhook-cert\") pod \"maas-controller-6d6f449ff5-jnvtz\" (UID: \"7fab7dcb-4329-4026-800c-2ea78618fcd7\") " pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:38.596958 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.596765 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gl8g\" (UniqueName: \"kubernetes.io/projected/7fab7dcb-4329-4026-800c-2ea78618fcd7-kube-api-access-5gl8g\") pod \"maas-controller-6d6f449ff5-jnvtz\" (UID: \"7fab7dcb-4329-4026-800c-2ea78618fcd7\") " pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:38.599420 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.599396 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fab7dcb-4329-4026-800c-2ea78618fcd7-webhook-cert\") pod \"maas-controller-6d6f449ff5-jnvtz\" (UID: \"7fab7dcb-4329-4026-800c-2ea78618fcd7\") " pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:38.604111 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.604087 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gl8g\" (UniqueName: \"kubernetes.io/projected/7fab7dcb-4329-4026-800c-2ea78618fcd7-kube-api-access-5gl8g\") pod \"maas-controller-6d6f449ff5-jnvtz\" (UID: \"7fab7dcb-4329-4026-800c-2ea78618fcd7\") " pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:38.664011 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.663978 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:38.795334 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.795308 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jnvtz"] Jul 06 01:38:38.796845 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:38:38.796820 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fab7dcb_4329_4026_800c_2ea78618fcd7.slice/crio-5af8eec541c588d2672cbbb7d73a15d3ab6dcecb49dd51920ddcb4ecfd2b2f5e WatchSource:0}: Error finding container 5af8eec541c588d2672cbbb7d73a15d3ab6dcecb49dd51920ddcb4ecfd2b2f5e: Status 404 returned error can't find the container with id 5af8eec541c588d2672cbbb7d73a15d3ab6dcecb49dd51920ddcb4ecfd2b2f5e Jul 06 01:38:38.899803 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.899780 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert\") pod \"maas-controller-6c8865c55d-z45zf\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:38.902251 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:38.902234 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert\") pod \"maas-controller-6c8865c55d-z45zf\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:39.082441 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:39.082415 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:39.235079 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:39.235056 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c8865c55d-z45zf"] Jul 06 01:38:39.236687 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:38:39.236662 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d382abe_586d_4bab_834a_3527cb48f041.slice/crio-716c5524725a438bff7f5ed886f9695aa68ed95f996f4fd166e2fd64068e5396 WatchSource:0}: Error finding container 716c5524725a438bff7f5ed886f9695aa68ed95f996f4fd166e2fd64068e5396: Status 404 returned error can't find the container with id 716c5524725a438bff7f5ed886f9695aa68ed95f996f4fd166e2fd64068e5396 Jul 06 01:38:39.383790 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:39.383661 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8865c55d-z45zf" event={"ID":"3d382abe-586d-4bab-834a-3527cb48f041","Type":"ContainerStarted","Data":"716c5524725a438bff7f5ed886f9695aa68ed95f996f4fd166e2fd64068e5396"} Jul 06 01:38:39.385127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:39.385095 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" event={"ID":"7fab7dcb-4329-4026-800c-2ea78618fcd7","Type":"ContainerStarted","Data":"5af8eec541c588d2672cbbb7d73a15d3ab6dcecb49dd51920ddcb4ecfd2b2f5e"} Jul 06 01:38:41.399336 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:41.399298 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8865c55d-z45zf" event={"ID":"3d382abe-586d-4bab-834a-3527cb48f041","Type":"ContainerStarted","Data":"cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170"} Jul 06 01:38:41.399754 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:41.399419 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:41.400658 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:41.400642 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" event={"ID":"7fab7dcb-4329-4026-800c-2ea78618fcd7","Type":"ContainerStarted","Data":"d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066"} Jul 06 01:38:41.400763 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:41.400749 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:41.421296 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:41.421247 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c8865c55d-z45zf" podStartSLOduration=1.8043188319999999 podStartE2EDuration="3.421233574s" totalImagesPullingTime="1.616914742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:38:41.419558661 +0000 UTC m=+719.554849166" watchObservedRunningTime="2026-07-06 01:38:41.421233574 +0000 UTC m=+719.556524077" Jul 06 01:38:41.439461 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:41.439421 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" podStartSLOduration=1.392563414 podStartE2EDuration="3.439411632s" totalImagesPullingTime="2.046848218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:38:41.437072647 +0000 UTC m=+719.572363150" watchObservedRunningTime="2026-07-06 01:38:41.439411632 +0000 UTC m=+719.574702136" Jul 06 01:38:52.411352 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.411318 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:38:52.411816 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.411733 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:52.470798 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.470754 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8865c55d-z45zf"] Jul 06 01:38:52.471242 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.471014 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6c8865c55d-z45zf" podUID="3d382abe-586d-4bab-834a-3527cb48f041" containerName="manager" containerID="cri-o://cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170" gracePeriod=30 Jul 06 01:38:52.716438 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.716416 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:52.814097 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.814075 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxcqv\" (UniqueName: \"kubernetes.io/projected/3d382abe-586d-4bab-834a-3527cb48f041-kube-api-access-vxcqv\") pod \"3d382abe-586d-4bab-834a-3527cb48f041\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " Jul 06 01:38:52.814248 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.814104 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert\") pod \"3d382abe-586d-4bab-834a-3527cb48f041\" (UID: \"3d382abe-586d-4bab-834a-3527cb48f041\") " Jul 06 01:38:52.816423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.816392 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d382abe-586d-4bab-834a-3527cb48f041-kube-api-access-vxcqv" (OuterVolumeSpecName: "kube-api-access-vxcqv") pod "3d382abe-586d-4bab-834a-3527cb48f041" (UID: "3d382abe-586d-4bab-834a-3527cb48f041"). InnerVolumeSpecName "kube-api-access-vxcqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:38:52.816423 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.816412 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3d382abe-586d-4bab-834a-3527cb48f041" (UID: "3d382abe-586d-4bab-834a-3527cb48f041"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:38:52.915242 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.915217 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxcqv\" (UniqueName: \"kubernetes.io/projected/3d382abe-586d-4bab-834a-3527cb48f041-kube-api-access-vxcqv\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:38:52.915242 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:52.915240 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3d382abe-586d-4bab-834a-3527cb48f041-webhook-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:38:53.466148 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.466107 2574 generic.go:356] "Generic (PLEG): container finished" podID="3d382abe-586d-4bab-834a-3527cb48f041" containerID="cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170" exitCode=0 Jul 06 01:38:53.466148 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.466139 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8865c55d-z45zf" event={"ID":"3d382abe-586d-4bab-834a-3527cb48f041","Type":"ContainerDied","Data":"cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170"} Jul 06 01:38:53.466587 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.466156 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c8865c55d-z45zf" Jul 06 01:38:53.466587 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.466198 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c8865c55d-z45zf" event={"ID":"3d382abe-586d-4bab-834a-3527cb48f041","Type":"ContainerDied","Data":"716c5524725a438bff7f5ed886f9695aa68ed95f996f4fd166e2fd64068e5396"} Jul 06 01:38:53.466587 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.466228 2574 scope.go:117] "RemoveContainer" containerID="cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170" Jul 06 01:38:53.475672 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.475485 2574 scope.go:117] "RemoveContainer" containerID="cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170" Jul 06 01:38:53.475777 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:38:53.475760 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170\": container with ID starting with cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170 not found: ID does not exist" containerID="cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170" Jul 06 01:38:53.475821 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.475786 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170"} err="failed to get container status \"cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170\": rpc error: code = NotFound desc = could not find container \"cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170\": container with ID starting with cd5ec4f843dff14b03cd2ab0d33964aeeb216fea6a65e3006c2f00f83bf4f170 not found: ID does not exist" Jul 06 01:38:53.485890 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.485870 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c8865c55d-z45zf"] Jul 06 01:38:53.488913 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:53.488895 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c8865c55d-z45zf"] Jul 06 01:38:54.382557 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:38:54.382529 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d382abe-586d-4bab-834a-3527cb48f041" path="/var/lib/kubelet/pods/3d382abe-586d-4bab-834a-3527cb48f041/volumes" Jul 06 01:39:18.086011 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.085976 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6ddd7f5dc5-b24gf"] Jul 06 01:39:18.086469 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.086455 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d382abe-586d-4bab-834a-3527cb48f041" containerName="manager" Jul 06 01:39:18.086521 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.086471 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d382abe-586d-4bab-834a-3527cb48f041" containerName="manager" Jul 06 01:39:18.086556 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.086551 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d382abe-586d-4bab-834a-3527cb48f041" containerName="manager" Jul 06 01:39:18.091799 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.091784 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.095794 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.095773 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 01:39:18.095940 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.095847 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-cfbht\"" Jul 06 01:39:18.102881 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.102838 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6ddd7f5dc5-b24gf"] Jul 06 01:39:18.232197 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.232165 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea7c9960-615e-4a5f-9011-3377d60cdd14-openshift-service-ca\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.232355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.232205 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea7c9960-615e-4a5f-9011-3377d60cdd14-maas-api-tls\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.232355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.232297 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgbvm\" (UniqueName: \"kubernetes.io/projected/ea7c9960-615e-4a5f-9011-3377d60cdd14-kube-api-access-kgbvm\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.333168 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.333138 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea7c9960-615e-4a5f-9011-3377d60cdd14-openshift-service-ca\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.333349 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.333178 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea7c9960-615e-4a5f-9011-3377d60cdd14-maas-api-tls\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.333349 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.333214 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgbvm\" (UniqueName: \"kubernetes.io/projected/ea7c9960-615e-4a5f-9011-3377d60cdd14-kube-api-access-kgbvm\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.333786 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.333766 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea7c9960-615e-4a5f-9011-3377d60cdd14-openshift-service-ca\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.335831 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.335807 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea7c9960-615e-4a5f-9011-3377d60cdd14-maas-api-tls\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.342663 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.342604 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgbvm\" (UniqueName: \"kubernetes.io/projected/ea7c9960-615e-4a5f-9011-3377d60cdd14-kube-api-access-kgbvm\") pod \"maas-api-6ddd7f5dc5-b24gf\" (UID: \"ea7c9960-615e-4a5f-9011-3377d60cdd14\") " pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.403475 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.403450 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:18.542680 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.542655 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6ddd7f5dc5-b24gf"] Jul 06 01:39:18.544464 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:39:18.544426 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea7c9960_615e_4a5f_9011_3377d60cdd14.slice/crio-ab24790f04698ae9c2df9e41e8f01dedc50f3a6411eee22b21f3da6f39b5bf2d WatchSource:0}: Error finding container ab24790f04698ae9c2df9e41e8f01dedc50f3a6411eee22b21f3da6f39b5bf2d: Status 404 returned error can't find the container with id ab24790f04698ae9c2df9e41e8f01dedc50f3a6411eee22b21f3da6f39b5bf2d Jul 06 01:39:18.604983 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.604908 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" event={"ID":"ea7c9960-615e-4a5f-9011-3377d60cdd14","Type":"ContainerStarted","Data":"ab24790f04698ae9c2df9e41e8f01dedc50f3a6411eee22b21f3da6f39b5bf2d"} Jul 06 01:39:18.666536 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.666506 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z9qkm"] Jul 06 01:39:18.671817 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.671800 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:18.673796 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.673777 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 01:39:18.673954 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.673776 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wjvkx\"" Jul 06 01:39:18.680264 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.680241 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z9qkm"] Jul 06 01:39:18.736333 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.736305 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b07852b5-6abf-422e-a7d0-699270afbb06-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z9qkm\" (UID: \"b07852b5-6abf-422e-a7d0-699270afbb06\") " pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:18.736448 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.736343 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rskbr\" (UniqueName: \"kubernetes.io/projected/b07852b5-6abf-422e-a7d0-699270afbb06-kube-api-access-rskbr\") pod \"payload-pre-processing-7964457696-z9qkm\" (UID: \"b07852b5-6abf-422e-a7d0-699270afbb06\") " pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:18.837167 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.837125 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b07852b5-6abf-422e-a7d0-699270afbb06-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z9qkm\" (UID: \"b07852b5-6abf-422e-a7d0-699270afbb06\") " pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:18.837350 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.837183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rskbr\" (UniqueName: \"kubernetes.io/projected/b07852b5-6abf-422e-a7d0-699270afbb06-kube-api-access-rskbr\") pod \"payload-pre-processing-7964457696-z9qkm\" (UID: \"b07852b5-6abf-422e-a7d0-699270afbb06\") " pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:18.837915 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.837851 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b07852b5-6abf-422e-a7d0-699270afbb06-plugins-config-volume\") pod \"payload-pre-processing-7964457696-z9qkm\" (UID: \"b07852b5-6abf-422e-a7d0-699270afbb06\") " pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:18.844363 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.844324 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rskbr\" (UniqueName: \"kubernetes.io/projected/b07852b5-6abf-422e-a7d0-699270afbb06-kube-api-access-rskbr\") pod \"payload-pre-processing-7964457696-z9qkm\" (UID: \"b07852b5-6abf-422e-a7d0-699270afbb06\") " pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:18.985180 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:18.985082 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:19.123447 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:19.123415 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-z9qkm"] Jul 06 01:39:19.123824 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:39:19.123626 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb07852b5_6abf_422e_a7d0_699270afbb06.slice/crio-6b232e86d3fc05fb681f2682ff79349400407308530c886274b547817be0cd64 WatchSource:0}: Error finding container 6b232e86d3fc05fb681f2682ff79349400407308530c886274b547817be0cd64: Status 404 returned error can't find the container with id 6b232e86d3fc05fb681f2682ff79349400407308530c886274b547817be0cd64 Jul 06 01:39:19.613968 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:19.613929 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" event={"ID":"b07852b5-6abf-422e-a7d0-699270afbb06","Type":"ContainerStarted","Data":"6b232e86d3fc05fb681f2682ff79349400407308530c886274b547817be0cd64"} Jul 06 01:39:21.628805 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:21.628770 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" event={"ID":"ea7c9960-615e-4a5f-9011-3377d60cdd14","Type":"ContainerStarted","Data":"4c47f2be3173f9ee63a24a6d746bb6e2143870af78cc139c1e37d9c2d2f533f9"} Jul 06 01:39:21.629296 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:21.628998 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:21.630309 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:21.630288 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" event={"ID":"b07852b5-6abf-422e-a7d0-699270afbb06","Type":"ContainerStarted","Data":"266d67697163df759c04412fdaa6d5a49b268c002e154c59bf52dc605d3f6800"} Jul 06 01:39:21.630413 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:21.630352 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:21.647964 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:21.647923 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" podStartSLOduration=1.468771216 podStartE2EDuration="3.64791051s" totalImagesPullingTime="2.179139294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:39:21.646538202 +0000 UTC m=+759.781828707" watchObservedRunningTime="2026-07-06 01:39:21.64791051 +0000 UTC m=+759.783201014" Jul 06 01:39:21.660667 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:21.660607 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" podStartSLOduration=2.059738592 podStartE2EDuration="3.660588197s" totalImagesPullingTime="1.600849605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:39:21.659975851 +0000 UTC m=+759.795266359" watchObservedRunningTime="2026-07-06 01:39:21.660588197 +0000 UTC m=+759.795878703" Jul 06 01:39:26.660637 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:26.660600 2574 generic.go:356] "Generic (PLEG): container finished" podID="abde296b-bf0d-45e4-8db4-d79ee2e60e0f" containerID="08a8f87b98e88bad8d299a50077a89114d5d20b763f72bbf66deee328d91fa0c" exitCode=0 Jul 06 01:39:26.661068 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:26.660678 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" event={"ID":"abde296b-bf0d-45e4-8db4-d79ee2e60e0f","Type":"ContainerDied","Data":"08a8f87b98e88bad8d299a50077a89114d5d20b763f72bbf66deee328d91fa0c"} Jul 06 01:39:26.661178 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:26.661163 2574 scope.go:117] "RemoveContainer" containerID="08a8f87b98e88bad8d299a50077a89114d5d20b763f72bbf66deee328d91fa0c" Jul 06 01:39:26.662314 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:26.662288 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:39:26.662374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:26.662359 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:39:26.662421 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:26.662400 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:39:27.138833 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:27.138806 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:39:27.641063 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:27.641033 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6ddd7f5dc5-b24gf" Jul 06 01:39:27.672297 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:27.672253 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" event={"ID":"abde296b-bf0d-45e4-8db4-d79ee2e60e0f","Type":"ContainerStarted","Data":"c85051933431edc90b16e0cc684db7804851fa9ea1291080fb5613ab74381e40"} Jul 06 01:39:28.063127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:28.063032 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:39:28.064436 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:28.064408 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-z8wf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 06 01:39:28.064665 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:28.064640 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" podUID="abde296b-bf0d-45e4-8db4-d79ee2e60e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 06 01:39:29.063758 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:29.063719 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-z8wf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 06 01:39:29.064359 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:29.064327 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" podUID="abde296b-bf0d-45e4-8db4-d79ee2e60e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 06 01:39:30.063238 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:30.063200 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-z8wf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 06 01:39:30.063519 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:30.063493 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" podUID="abde296b-bf0d-45e4-8db4-d79ee2e60e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 06 01:39:31.063565 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:31.063531 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-z8wf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 06 01:39:31.064181 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:31.064143 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" podUID="abde296b-bf0d-45e4-8db4-d79ee2e60e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 06 01:39:32.067003 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:32.066960 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:39:32.067468 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:32.067330 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:39:32.068081 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:32.068066 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-z8wf7" Jul 06 01:39:32.637226 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:32.637186 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" podUID="b07852b5-6abf-422e-a7d0-699270afbb06" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.48:9004: connect: connection refused" Jul 06 01:39:40.835982 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.835952 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb"] Jul 06 01:39:40.838749 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.838725 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:40.841741 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.841719 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 01:39:40.841852 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.841719 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 01:39:40.841852 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.841720 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cxdt6\"" Jul 06 01:39:40.841852 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.841728 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 01:39:40.848802 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.848778 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb"] Jul 06 01:39:40.951706 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.951678 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:40.951891 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.951720 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:40.951891 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.951752 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:40.951891 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.951794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf5r8\" (UniqueName: \"kubernetes.io/projected/ad343c45-028e-45eb-a728-f379f458f310-kube-api-access-vf5r8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:40.951891 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.951818 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad343c45-028e-45eb-a728-f379f458f310-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:40.951891 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:40.951851 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.052982 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.052953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.053177 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.052996 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.053177 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.053015 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.053177 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.053057 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf5r8\" (UniqueName: \"kubernetes.io/projected/ad343c45-028e-45eb-a728-f379f458f310-kube-api-access-vf5r8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.053177 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.053080 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad343c45-028e-45eb-a728-f379f458f310-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.053177 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.053117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.053443 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.053417 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.053525 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.053501 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.053812 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.053789 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.055571 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.055551 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ad343c45-028e-45eb-a728-f379f458f310-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.056442 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.056418 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ad343c45-028e-45eb-a728-f379f458f310-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.061375 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.061351 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf5r8\" (UniqueName: \"kubernetes.io/projected/ad343c45-028e-45eb-a728-f379f458f310-kube-api-access-vf5r8\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb\" (UID: \"ad343c45-028e-45eb-a728-f379f458f310\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.150846 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.150771 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:41.288976 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.288951 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb"] Jul 06 01:39:41.290667 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:39:41.290641 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad343c45_028e_45eb_a728_f379f458f310.slice/crio-ca764296c076b7daf6dead7891e94bb9096b00563177b08e19cd5d043655df62 WatchSource:0}: Error finding container ca764296c076b7daf6dead7891e94bb9096b00563177b08e19cd5d043655df62: Status 404 returned error can't find the container with id ca764296c076b7daf6dead7891e94bb9096b00563177b08e19cd5d043655df62 Jul 06 01:39:41.756735 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:41.756645 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" event={"ID":"ad343c45-028e-45eb-a728-f379f458f310","Type":"ContainerStarted","Data":"ca764296c076b7daf6dead7891e94bb9096b00563177b08e19cd5d043655df62"} Jul 06 01:39:42.636655 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:42.636571 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-z9qkm" Jul 06 01:39:42.764185 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:42.764151 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" event={"ID":"ad343c45-028e-45eb-a728-f379f458f310","Type":"ContainerStarted","Data":"97da0bce1ac01fe975a8dfc65d3ca486881355f68e0f9ae6f9109e859e82d1c8"} Jul 06 01:39:42.780241 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:42.780192 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" podStartSLOduration=1.643897084 podStartE2EDuration="2.780178224s" totalImagesPullingTime="1.13628114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:39:42.778350582 +0000 UTC m=+780.913641087" watchObservedRunningTime="2026-07-06 01:39:42.780178224 +0000 UTC m=+780.915468784" Jul 06 01:39:51.151489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:51.151453 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:51.151942 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:51.151511 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:51.167756 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:51.167730 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:39:51.829133 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:39:51.829107 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb" Jul 06 01:40:10.522382 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.522351 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6"] Jul 06 01:40:10.526032 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.526013 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.528106 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.528089 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 01:40:10.535798 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.535774 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6"] Jul 06 01:40:10.622684 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.622653 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.622881 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.622691 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.622881 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.622792 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b12ac32-0034-452d-9656-927d3ba0a65a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.623023 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.622909 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.623023 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.622938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.623023 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.622966 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lnbf\" (UniqueName: \"kubernetes.io/projected/7b12ac32-0034-452d-9656-927d3ba0a65a-kube-api-access-5lnbf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.723774 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.723736 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b12ac32-0034-452d-9656-927d3ba0a65a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.723981 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.723850 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.723981 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.723896 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.723981 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.723921 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lnbf\" (UniqueName: \"kubernetes.io/projected/7b12ac32-0034-452d-9656-927d3ba0a65a-kube-api-access-5lnbf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.723981 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.723975 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.724216 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.723999 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.724337 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.724263 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.724638 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.724606 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.724845 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.724820 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.726909 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.726836 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b12ac32-0034-452d-9656-927d3ba0a65a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.727435 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.727413 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b12ac32-0034-452d-9656-927d3ba0a65a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.731957 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.731937 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lnbf\" (UniqueName: \"kubernetes.io/projected/7b12ac32-0034-452d-9656-927d3ba0a65a-kube-api-access-5lnbf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-grxs6\" (UID: \"7b12ac32-0034-452d-9656-927d3ba0a65a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.836324 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.836296 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:10.983956 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:10.983926 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6"] Jul 06 01:40:10.984736 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:40:10.984712 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b12ac32_0034_452d_9656_927d3ba0a65a.slice/crio-4235778220a104f4057ade88e35c6d6121083a8a2e68c48fe909ce06aff176b9 WatchSource:0}: Error finding container 4235778220a104f4057ade88e35c6d6121083a8a2e68c48fe909ce06aff176b9: Status 404 returned error can't find the container with id 4235778220a104f4057ade88e35c6d6121083a8a2e68c48fe909ce06aff176b9 Jul 06 01:40:11.934762 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:11.934725 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" event={"ID":"7b12ac32-0034-452d-9656-927d3ba0a65a","Type":"ContainerStarted","Data":"d8229dbd64c586a4285875949bf0707a952a6037ea0e199536e56b463bad241d"} Jul 06 01:40:11.935343 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:11.934820 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" event={"ID":"7b12ac32-0034-452d-9656-927d3ba0a65a","Type":"ContainerStarted","Data":"4235778220a104f4057ade88e35c6d6121083a8a2e68c48fe909ce06aff176b9"} Jul 06 01:40:11.952457 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:11.952413 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" podStartSLOduration=1.7756790649999998 podStartE2EDuration="1.952398066s" totalImagesPullingTime="176.719001ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:40:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:40:11.949827381 +0000 UTC m=+810.085117886" watchObservedRunningTime="2026-07-06 01:40:11.952398066 +0000 UTC m=+810.087688569" Jul 06 01:40:20.836906 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:20.836866 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:20.837385 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:20.836933 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:20.849158 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:20.849137 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:40:20.998621 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:40:20.998598 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-grxs6" Jul 06 01:41:42.331542 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:41:42.331470 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:41:42.333516 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:41:42.333471 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:41:42.339143 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:41:42.339123 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:41:42.340493 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:41:42.340476 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:42:43.011613 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.011574 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jnvtz"] Jul 06 01:42:43.012128 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.011878 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" podUID="7fab7dcb-4329-4026-800c-2ea78618fcd7" containerName="manager" containerID="cri-o://d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066" gracePeriod=30 Jul 06 01:42:43.256652 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.256630 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:42:43.397104 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.397068 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fab7dcb-4329-4026-800c-2ea78618fcd7-webhook-cert\") pod \"7fab7dcb-4329-4026-800c-2ea78618fcd7\" (UID: \"7fab7dcb-4329-4026-800c-2ea78618fcd7\") " Jul 06 01:42:43.397295 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.397137 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gl8g\" (UniqueName: \"kubernetes.io/projected/7fab7dcb-4329-4026-800c-2ea78618fcd7-kube-api-access-5gl8g\") pod \"7fab7dcb-4329-4026-800c-2ea78618fcd7\" (UID: \"7fab7dcb-4329-4026-800c-2ea78618fcd7\") " Jul 06 01:42:43.399284 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.399253 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fab7dcb-4329-4026-800c-2ea78618fcd7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7fab7dcb-4329-4026-800c-2ea78618fcd7" (UID: "7fab7dcb-4329-4026-800c-2ea78618fcd7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 01:42:43.399378 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.399360 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fab7dcb-4329-4026-800c-2ea78618fcd7-kube-api-access-5gl8g" (OuterVolumeSpecName: "kube-api-access-5gl8g") pod "7fab7dcb-4329-4026-800c-2ea78618fcd7" (UID: "7fab7dcb-4329-4026-800c-2ea78618fcd7"). InnerVolumeSpecName "kube-api-access-5gl8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:42:43.498147 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.498124 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7fab7dcb-4329-4026-800c-2ea78618fcd7-webhook-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:42:43.498147 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.498145 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gl8g\" (UniqueName: \"kubernetes.io/projected/7fab7dcb-4329-4026-800c-2ea78618fcd7-kube-api-access-5gl8g\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:42:43.799255 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.799173 2574 generic.go:356] "Generic (PLEG): container finished" podID="7fab7dcb-4329-4026-800c-2ea78618fcd7" containerID="d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066" exitCode=0 Jul 06 01:42:43.799255 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.799234 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" Jul 06 01:42:43.799437 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.799259 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" event={"ID":"7fab7dcb-4329-4026-800c-2ea78618fcd7","Type":"ContainerDied","Data":"d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066"} Jul 06 01:42:43.799437 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.799318 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d6f449ff5-jnvtz" event={"ID":"7fab7dcb-4329-4026-800c-2ea78618fcd7","Type":"ContainerDied","Data":"5af8eec541c588d2672cbbb7d73a15d3ab6dcecb49dd51920ddcb4ecfd2b2f5e"} Jul 06 01:42:43.799437 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.799345 2574 scope.go:117] "RemoveContainer" containerID="d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066" Jul 06 01:42:43.809400 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.809381 2574 scope.go:117] "RemoveContainer" containerID="d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066" Jul 06 01:42:43.809671 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:42:43.809652 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066\": container with ID starting with d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066 not found: ID does not exist" containerID="d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066" Jul 06 01:42:43.809747 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.809680 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066"} err="failed to get container status \"d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066\": rpc error: code = NotFound desc = could not find container \"d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066\": container with ID starting with d944ca90b55fbd50dd1bd8b30b3dc9a3bdb7643bdfb562acf6de7c0cf4ed8066 not found: ID does not exist" Jul 06 01:42:43.825537 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.825518 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jnvtz"] Jul 06 01:42:43.831342 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:43.831321 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jnvtz"] Jul 06 01:42:44.382591 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.382562 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fab7dcb-4329-4026-800c-2ea78618fcd7" path="/var/lib/kubelet/pods/7fab7dcb-4329-4026-800c-2ea78618fcd7/volumes" Jul 06 01:42:44.689593 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.689511 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jszv6"] Jul 06 01:42:44.690024 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.690007 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fab7dcb-4329-4026-800c-2ea78618fcd7" containerName="manager" Jul 06 01:42:44.690078 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.690026 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fab7dcb-4329-4026-800c-2ea78618fcd7" containerName="manager" Jul 06 01:42:44.690141 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.690130 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fab7dcb-4329-4026-800c-2ea78618fcd7" containerName="manager" Jul 06 01:42:44.694605 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.694586 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:44.697627 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.697609 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dr8hv\"" Jul 06 01:42:44.697697 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.697640 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 01:42:44.697697 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.697679 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 01:42:44.705457 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.705437 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jszv6"] Jul 06 01:42:44.810599 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.810567 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/951492ed-9116-467d-ab29-d0cb52e555dd-webhook-cert\") pod \"maas-controller-6d6f449ff5-jszv6\" (UID: \"951492ed-9116-467d-ab29-d0cb52e555dd\") " pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:44.810794 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.810645 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjj8r\" (UniqueName: \"kubernetes.io/projected/951492ed-9116-467d-ab29-d0cb52e555dd-kube-api-access-jjj8r\") pod \"maas-controller-6d6f449ff5-jszv6\" (UID: \"951492ed-9116-467d-ab29-d0cb52e555dd\") " pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:44.911676 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.911643 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/951492ed-9116-467d-ab29-d0cb52e555dd-webhook-cert\") pod \"maas-controller-6d6f449ff5-jszv6\" (UID: \"951492ed-9116-467d-ab29-d0cb52e555dd\") " pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:44.911834 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.911700 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjj8r\" (UniqueName: \"kubernetes.io/projected/951492ed-9116-467d-ab29-d0cb52e555dd-kube-api-access-jjj8r\") pod \"maas-controller-6d6f449ff5-jszv6\" (UID: \"951492ed-9116-467d-ab29-d0cb52e555dd\") " pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:44.914227 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.914201 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/951492ed-9116-467d-ab29-d0cb52e555dd-webhook-cert\") pod \"maas-controller-6d6f449ff5-jszv6\" (UID: \"951492ed-9116-467d-ab29-d0cb52e555dd\") " pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:44.921981 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:44.921958 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjj8r\" (UniqueName: \"kubernetes.io/projected/951492ed-9116-467d-ab29-d0cb52e555dd-kube-api-access-jjj8r\") pod \"maas-controller-6d6f449ff5-jszv6\" (UID: \"951492ed-9116-467d-ab29-d0cb52e555dd\") " pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:45.006042 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:45.005974 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:45.148843 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:45.148816 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d6f449ff5-jszv6"] Jul 06 01:42:45.150886 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:42:45.150846 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951492ed_9116_467d_ab29_d0cb52e555dd.slice/crio-8234e70324d27387d8d3cad0a37e0d19053d65d9964b0566e5189684d3aaf803 WatchSource:0}: Error finding container 8234e70324d27387d8d3cad0a37e0d19053d65d9964b0566e5189684d3aaf803: Status 404 returned error can't find the container with id 8234e70324d27387d8d3cad0a37e0d19053d65d9964b0566e5189684d3aaf803 Jul 06 01:42:45.152181 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:45.152162 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 01:42:45.816331 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:45.816245 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d6f449ff5-jszv6" event={"ID":"951492ed-9116-467d-ab29-d0cb52e555dd","Type":"ContainerStarted","Data":"d76b4ccd73d63bda4b0450d60509fd58fdabcdb4c874600a0534ef9d83996120"} Jul 06 01:42:45.816331 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:45.816295 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d6f449ff5-jszv6" event={"ID":"951492ed-9116-467d-ab29-d0cb52e555dd","Type":"ContainerStarted","Data":"8234e70324d27387d8d3cad0a37e0d19053d65d9964b0566e5189684d3aaf803"} Jul 06 01:42:45.816715 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:45.816343 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:42:45.847934 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:45.847890 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d6f449ff5-jszv6" podStartSLOduration=1.49292618 podStartE2EDuration="1.847875721s" totalImagesPullingTime="354.949541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:42:45.845725889 +0000 UTC m=+963.981016394" watchObservedRunningTime="2026-07-06 01:42:45.847875721 +0000 UTC m=+963.983166222" Jul 06 01:42:56.828736 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:42:56.828706 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d6f449ff5-jszv6" Jul 06 01:46:42.376023 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:46:42.375992 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:46:42.380568 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:46:42.380532 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:46:42.385046 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:46:42.385026 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:46:42.389717 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:46:42.389702 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:51:42.416285 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:51:42.416254 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:51:42.423221 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:51:42.423194 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:51:42.423812 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:51:42.423792 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:51:42.429589 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:51:42.429574 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:54:49.150077 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.150043 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb"] Jul 06 01:54:49.154312 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.154292 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.156355 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.156333 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-adaa0b9e-openshift-default-dockercfg-ch2n5\"" Jul 06 01:54:49.170657 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.170637 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb"] Jul 06 01:54:49.286350 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286316 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwpnq\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-kube-api-access-mwpnq\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.286350 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286353 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-socket\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.286553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286378 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-credential-socket\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.286553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286443 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-data\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.286553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286519 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-certs\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.286553 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286548 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-envoy\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.286689 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286570 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab732b53-5078-49ae-92f4-52d8cff46e96-istiod-ca-cert\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.286689 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286603 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-podinfo\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.286689 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.286645 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-token\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.387551 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387516 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-envoy\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.387711 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387567 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab732b53-5078-49ae-92f4-52d8cff46e96-istiod-ca-cert\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.387711 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387592 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-podinfo\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.387711 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387615 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-token\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.387711 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387662 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwpnq\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-kube-api-access-mwpnq\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.387711 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387682 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-socket\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.387711 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387710 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-credential-socket\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.388083 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387735 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-data\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.388083 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.387807 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-certs\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.388264 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.388234 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-credential-socket\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.388560 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.388325 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-socket\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.388560 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.388361 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-data\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.388560 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.388426 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-certs\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.388560 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.388457 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab732b53-5078-49ae-92f4-52d8cff46e96-istiod-ca-cert\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.390092 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.390073 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-envoy\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.390304 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.390289 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-podinfo\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.395655 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.395633 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-token\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.395740 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.395661 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwpnq\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-kube-api-access-mwpnq\") pod \"e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.465340 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.465271 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:49.607969 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.607940 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb"] Jul 06 01:54:49.609066 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:54:49.609044 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab732b53_5078_49ae_92f4_52d8cff46e96.slice/crio-42f7707ceef585277a08f8762bcd047c6617936f2d52cbe58ed1c13b8acfbb9f WatchSource:0}: Error finding container 42f7707ceef585277a08f8762bcd047c6617936f2d52cbe58ed1c13b8acfbb9f: Status 404 returned error can't find the container with id 42f7707ceef585277a08f8762bcd047c6617936f2d52cbe58ed1c13b8acfbb9f Jul 06 01:54:49.611684 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.611653 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:54:49.611741 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.611717 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:54:49.611776 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.611748 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:54:49.903825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.903788 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" event={"ID":"ab732b53-5078-49ae-92f4-52d8cff46e96","Type":"ContainerStarted","Data":"b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e"} Jul 06 01:54:49.904034 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:49.903872 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" event={"ID":"ab732b53-5078-49ae-92f4-52d8cff46e96","Type":"ContainerStarted","Data":"42f7707ceef585277a08f8762bcd047c6617936f2d52cbe58ed1c13b8acfbb9f"} Jul 06 01:54:50.465560 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:50.465526 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:50.471461 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:50.471427 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:50.500565 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:50.500519 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" podStartSLOduration=1.5005054690000001 podStartE2EDuration="1.500505469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:54:49.944669756 +0000 UTC m=+1688.079960259" watchObservedRunningTime="2026-07-06 01:54:50.500505469 +0000 UTC m=+1688.635795973" Jul 06 01:54:50.909388 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:50.909359 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:50.910375 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:50.910354 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:54:55.735683 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:55.735646 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb"] Jul 06 01:54:55.736181 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:54:55.735881 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" podUID="ab732b53-5078-49ae-92f4-52d8cff46e96" containerName="istio-proxy" containerID="cri-o://b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e" gracePeriod=30 Jul 06 01:55:02.197181 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.197122 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:55:02.270220 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.270182 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n"] Jul 06 01:55:02.270626 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.270611 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab732b53-5078-49ae-92f4-52d8cff46e96" containerName="istio-proxy" Jul 06 01:55:02.270626 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.270626 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab732b53-5078-49ae-92f4-52d8cff46e96" containerName="istio-proxy" Jul 06 01:55:02.270769 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.270699 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab732b53-5078-49ae-92f4-52d8cff46e96" containerName="istio-proxy" Jul 06 01:55:02.272815 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.272793 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.278171 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.278132 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d5d69b37-gw-openshift-default-dockercfg-gb2q7\"" Jul 06 01:55:02.325376 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.325341 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n"] Jul 06 01:55:02.325683 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.325660 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab732b53-5078-49ae-92f4-52d8cff46e96-istiod-ca-cert\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.325734 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.325707 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwpnq\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-kube-api-access-mwpnq\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.325779 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.325736 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-podinfo\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.325779 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.325766 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-socket\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.325898 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.325791 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-token\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.325898 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.325843 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-credential-socket\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.326034 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.325896 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-envoy\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.326034 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326004 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-data\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.326152 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326057 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-certs\") pod \"ab732b53-5078-49ae-92f4-52d8cff46e96\" (UID: \"ab732b53-5078-49ae-92f4-52d8cff46e96\") " Jul 06 01:55:02.326227 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326193 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:02.326227 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326197 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab732b53-5078-49ae-92f4-52d8cff46e96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:55:02.327055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326421 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:02.327055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326609 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:02.327055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326741 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ab732b53-5078-49ae-92f4-52d8cff46e96-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.327055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326763 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.327055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326776 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.327055 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.326790 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.327666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.327183 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-data" (OuterVolumeSpecName: "istio-data") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:02.328782 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.328756 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:02.329099 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.329043 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 01:55:02.329178 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.329112 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-kube-api-access-mwpnq" (OuterVolumeSpecName: "kube-api-access-mwpnq") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "kube-api-access-mwpnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:55:02.329178 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.329131 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-token" (OuterVolumeSpecName: "istio-token") pod "ab732b53-5078-49ae-92f4-52d8cff46e96" (UID: "ab732b53-5078-49ae-92f4-52d8cff46e96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:55:02.428078 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428048 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-credential-socket\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428247 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428081 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-data\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428247 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-token\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428247 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428133 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-envoy\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428247 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428182 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce8a688e-7ae2-477d-92a1-d0315cd06429-istiod-ca-cert\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428247 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-socket\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428465 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428281 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-podinfo\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428465 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428319 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbztv\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-kube-api-access-jbztv\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428465 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428407 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-certs\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.428568 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428471 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwpnq\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-kube-api-access-mwpnq\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.428568 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428482 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.428568 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428492 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.428568 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428503 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.428568 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.428512 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ab732b53-5078-49ae-92f4-52d8cff46e96-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:02.529611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529529 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-podinfo\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.529611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529567 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbztv\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-kube-api-access-jbztv\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.529611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529607 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-certs\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.529936 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529654 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-credential-socket\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.529936 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529683 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-data\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.529936 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529705 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-token\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.529936 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529781 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-envoy\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.529936 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529808 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce8a688e-7ae2-477d-92a1-d0315cd06429-istiod-ca-cert\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.529936 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.529894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-socket\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.530252 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.530146 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-credential-socket\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.530252 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.530175 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-certs\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.530543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.530325 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-socket\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.530543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.530356 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-data\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.530704 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.530684 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce8a688e-7ae2-477d-92a1-d0315cd06429-istiod-ca-cert\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.532194 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.532165 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-envoy\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.532290 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.532215 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-podinfo\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.540266 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.540243 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-token\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.541008 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.540983 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbztv\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-kube-api-access-jbztv\") pod \"e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.593507 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.593482 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:02.733821 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.733797 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n"] Jul 06 01:55:02.735906 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:55:02.735880 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8a688e_7ae2_477d_92a1_d0315cd06429.slice/crio-1ef09003aa4bd81f76e654c310e54a48f84e1751665c632f3a2e5001519be7b4 WatchSource:0}: Error finding container 1ef09003aa4bd81f76e654c310e54a48f84e1751665c632f3a2e5001519be7b4: Status 404 returned error can't find the container with id 1ef09003aa4bd81f76e654c310e54a48f84e1751665c632f3a2e5001519be7b4 Jul 06 01:55:02.737919 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.737852 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:02.738025 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.737962 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:02.738025 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.738006 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:02.983359 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.983329 2574 generic.go:356] "Generic (PLEG): container finished" podID="ab732b53-5078-49ae-92f4-52d8cff46e96" containerID="b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e" exitCode=0 Jul 06 01:55:02.983527 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.983418 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" Jul 06 01:55:02.983611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.983416 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" event={"ID":"ab732b53-5078-49ae-92f4-52d8cff46e96","Type":"ContainerDied","Data":"b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e"} Jul 06 01:55:02.983611 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.983600 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb" event={"ID":"ab732b53-5078-49ae-92f4-52d8cff46e96","Type":"ContainerDied","Data":"42f7707ceef585277a08f8762bcd047c6617936f2d52cbe58ed1c13b8acfbb9f"} Jul 06 01:55:02.983702 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.983635 2574 scope.go:117] "RemoveContainer" containerID="b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e" Jul 06 01:55:02.985556 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.985528 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" event={"ID":"ce8a688e-7ae2-477d-92a1-d0315cd06429","Type":"ContainerStarted","Data":"bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0"} Jul 06 01:55:02.985689 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.985571 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" event={"ID":"ce8a688e-7ae2-477d-92a1-d0315cd06429","Type":"ContainerStarted","Data":"1ef09003aa4bd81f76e654c310e54a48f84e1751665c632f3a2e5001519be7b4"} Jul 06 01:55:02.995248 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.995230 2574 scope.go:117] "RemoveContainer" containerID="b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e" Jul 06 01:55:02.995535 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:55:02.995518 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e\": container with ID starting with b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e not found: ID does not exist" containerID="b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e" Jul 06 01:55:02.995609 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:02.995545 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e"} err="failed to get container status \"b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e\": rpc error: code = NotFound desc = could not find container \"b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e\": container with ID starting with b0632c7faa7eb12d2454eac351192ff648c17bcb5e7d5be28d0749973ead421e not found: ID does not exist" Jul 06 01:55:03.006260 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:03.006213 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" podStartSLOduration=1.006197918 podStartE2EDuration="1.006197918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:55:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:55:03.00409056 +0000 UTC m=+1701.139381064" watchObservedRunningTime="2026-07-06 01:55:03.006197918 +0000 UTC m=+1701.141488423" Jul 06 01:55:03.019705 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:03.019682 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb"] Jul 06 01:55:03.023314 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:03.023267 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-adaa0b9e-openshift-default-684cffffc-kf8nb"] Jul 06 01:55:03.593994 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:03.593914 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:03.598955 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:03.598932 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:03.993046 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:03.992969 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:03.993962 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:03.993946 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:04.382690 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:04.382655 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab732b53-5078-49ae-92f4-52d8cff46e96" path="/var/lib/kubelet/pods/ab732b53-5078-49ae-92f4-52d8cff46e96/volumes" Jul 06 01:55:08.369396 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:08.369362 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n"] Jul 06 01:55:08.369814 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:08.369537 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" podUID="ce8a688e-7ae2-477d-92a1-d0315cd06429" containerName="istio-proxy" containerID="cri-o://bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0" gracePeriod=30 Jul 06 01:55:13.634677 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.634619 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:13.734297 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734269 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbztv\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-kube-api-access-jbztv\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734297 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734300 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-token\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734534 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734345 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-socket\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734534 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734400 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-envoy\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734534 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734431 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-certs\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734534 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734464 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-podinfo\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734534 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734495 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce8a688e-7ae2-477d-92a1-d0315cd06429-istiod-ca-cert\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734534 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734518 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-data\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734877 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734559 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-credential-socket\") pod \"ce8a688e-7ae2-477d-92a1-d0315cd06429\" (UID: \"ce8a688e-7ae2-477d-92a1-d0315cd06429\") " Jul 06 01:55:13.734877 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734652 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:13.734877 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734769 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:13.734877 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734827 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-data" (OuterVolumeSpecName: "istio-data") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:13.735159 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734968 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:13.735159 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.734987 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:13.735159 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.735001 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:13.735159 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.735051 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:13.735159 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.735166 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce8a688e-7ae2-477d-92a1-d0315cd06429-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:55:13.736781 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.736750 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:13.736781 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.736750 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-kube-api-access-jbztv" (OuterVolumeSpecName: "kube-api-access-jbztv") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "kube-api-access-jbztv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:55:13.737164 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.737142 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-token" (OuterVolumeSpecName: "istio-token") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:55:13.737343 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.737316 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ce8a688e-7ae2-477d-92a1-d0315cd06429" (UID: "ce8a688e-7ae2-477d-92a1-d0315cd06429"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 01:55:13.835799 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.835766 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:13.835799 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.835791 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:13.835799 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.835800 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce8a688e-7ae2-477d-92a1-d0315cd06429-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:13.835799 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.835810 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce8a688e-7ae2-477d-92a1-d0315cd06429-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:13.836082 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.835818 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbztv\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-kube-api-access-jbztv\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:13.836082 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:13.835827 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce8a688e-7ae2-477d-92a1-d0315cd06429-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:14.054108 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.054023 2574 generic.go:356] "Generic (PLEG): container finished" podID="ce8a688e-7ae2-477d-92a1-d0315cd06429" containerID="bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0" exitCode=0 Jul 06 01:55:14.054108 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.054070 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" Jul 06 01:55:14.054345 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.054116 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" event={"ID":"ce8a688e-7ae2-477d-92a1-d0315cd06429","Type":"ContainerDied","Data":"bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0"} Jul 06 01:55:14.054345 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.054174 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n" event={"ID":"ce8a688e-7ae2-477d-92a1-d0315cd06429","Type":"ContainerDied","Data":"1ef09003aa4bd81f76e654c310e54a48f84e1751665c632f3a2e5001519be7b4"} Jul 06 01:55:14.054345 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.054202 2574 scope.go:117] "RemoveContainer" containerID="bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0" Jul 06 01:55:14.064250 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.064235 2574 scope.go:117] "RemoveContainer" containerID="bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0" Jul 06 01:55:14.064505 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:55:14.064478 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0\": container with ID starting with bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0 not found: ID does not exist" containerID="bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0" Jul 06 01:55:14.064555 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.064514 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0"} err="failed to get container status \"bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0\": rpc error: code = NotFound desc = could not find container \"bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0\": container with ID starting with bc5ad0c71d8dd168385eb2bc75b92e51b7dd3d53efeded0af7ae515ac0e7ffc0 not found: ID does not exist" Jul 06 01:55:14.076020 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.075999 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n"] Jul 06 01:55:14.081411 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.081392 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d5d69b37-gw-openshift-default-7f77c4b95-clb8n"] Jul 06 01:55:14.383137 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.383107 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce8a688e-7ae2-477d-92a1-d0315cd06429" path="/var/lib/kubelet/pods/ce8a688e-7ae2-477d-92a1-d0315cd06429/volumes" Jul 06 01:55:14.574333 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.574305 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj"] Jul 06 01:55:14.574794 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.574777 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce8a688e-7ae2-477d-92a1-d0315cd06429" containerName="istio-proxy" Jul 06 01:55:14.574883 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.574796 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce8a688e-7ae2-477d-92a1-d0315cd06429" containerName="istio-proxy" Jul 06 01:55:14.574939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.574899 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce8a688e-7ae2-477d-92a1-d0315cd06429" containerName="istio-proxy" Jul 06 01:55:14.579353 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.579336 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.581668 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.581649 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a25d8b7a-openshift-default-dockercfg-wb8qh\"" Jul 06 01:55:14.601993 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.597390 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj"] Jul 06 01:55:14.643459 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-token\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.643459 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643450 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgmv8\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-kube-api-access-dgmv8\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.643825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643484 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-envoy\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.643825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643501 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-data\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.643825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643522 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-socket\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.643825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ead7268e-c4a2-41d1-a900-3aae7e924535-istiod-ca-cert\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.643825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643647 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-certs\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.643825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-credential-socket\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.643825 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.643682 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-podinfo\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745082 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745042 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-certs\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745082 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745083 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-credential-socket\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745334 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745099 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-podinfo\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745334 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745131 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-token\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745334 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745315 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgmv8\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-kube-api-access-dgmv8\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745499 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745361 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-envoy\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745499 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745390 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-data\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745499 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-socket\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745641 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745504 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-credential-socket\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745641 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745540 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-certs\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745641 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745555 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ead7268e-c4a2-41d1-a900-3aae7e924535-istiod-ca-cert\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745908 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745882 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-data\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.745908 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.745907 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-socket\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.746482 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.746459 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ead7268e-c4a2-41d1-a900-3aae7e924535-istiod-ca-cert\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.747694 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.747673 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-envoy\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.748006 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.747986 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-podinfo\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.753731 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.753703 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-token\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.753920 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.753903 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgmv8\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-kube-api-access-dgmv8\") pod \"e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:14.889826 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:14.889796 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:15.031951 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:15.031919 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj"] Jul 06 01:55:15.035647 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:55:15.035616 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podead7268e_c4a2_41d1_a900_3aae7e924535.slice/crio-f692ae16502f6f92df7f1e5aebbe3026c53b32c2e983c9552fa58f2ca3db297d WatchSource:0}: Error finding container f692ae16502f6f92df7f1e5aebbe3026c53b32c2e983c9552fa58f2ca3db297d: Status 404 returned error can't find the container with id f692ae16502f6f92df7f1e5aebbe3026c53b32c2e983c9552fa58f2ca3db297d Jul 06 01:55:15.037962 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:15.037924 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:15.038046 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:15.037995 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:15.038046 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:15.038023 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:15.064104 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:15.064080 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" event={"ID":"ead7268e-c4a2-41d1-a900-3aae7e924535","Type":"ContainerStarted","Data":"f692ae16502f6f92df7f1e5aebbe3026c53b32c2e983c9552fa58f2ca3db297d"} Jul 06 01:55:16.073326 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:16.073285 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" event={"ID":"ead7268e-c4a2-41d1-a900-3aae7e924535","Type":"ContainerStarted","Data":"1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86"} Jul 06 01:55:16.096539 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:16.096473 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" podStartSLOduration=2.096453901 podStartE2EDuration="2.096453901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:55:16.095735117 +0000 UTC m=+1714.231025620" watchObservedRunningTime="2026-07-06 01:55:16.096453901 +0000 UTC m=+1714.231744406" Jul 06 01:55:16.890738 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:16.890704 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:16.895236 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:16.895216 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:17.082116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:17.082087 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:17.083072 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:17.083049 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:27.447077 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:27.447044 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj"] Jul 06 01:55:27.447603 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:27.447234 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" podUID="ead7268e-c4a2-41d1-a900-3aae7e924535" containerName="istio-proxy" containerID="cri-o://1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86" gracePeriod=30 Jul 06 01:55:32.084421 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.084377 2574 patch_prober.go:28] interesting pod/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 01:55:32.084788 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.084451 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" podUID="ead7268e-c4a2-41d1-a900-3aae7e924535" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 01:55:32.692159 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.692130 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:32.811991 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.811958 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-token\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812162 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812030 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-credential-socket\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812162 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812047 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-podinfo\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812162 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812082 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgmv8\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-kube-api-access-dgmv8\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812162 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812110 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ead7268e-c4a2-41d1-a900-3aae7e924535-istiod-ca-cert\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812162 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812137 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-certs\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812431 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812201 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-envoy\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812431 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812224 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-socket\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812431 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812255 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-data\") pod \"ead7268e-c4a2-41d1-a900-3aae7e924535\" (UID: \"ead7268e-c4a2-41d1-a900-3aae7e924535\") " Jul 06 01:55:32.812431 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812368 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:32.812646 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812466 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:32.812646 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812581 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:32.812646 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812628 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ead7268e-c4a2-41d1-a900-3aae7e924535-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:55:32.813007 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812644 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:32.813007 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812664 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:32.813007 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812678 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:32.813007 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.812763 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-data" (OuterVolumeSpecName: "istio-data") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:32.814371 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.814344 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-kube-api-access-dgmv8" (OuterVolumeSpecName: "kube-api-access-dgmv8") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "kube-api-access-dgmv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:55:32.814489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.814371 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-token" (OuterVolumeSpecName: "istio-token") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:55:32.814489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.814417 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 01:55:32.814567 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.814487 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ead7268e-c4a2-41d1-a900-3aae7e924535" (UID: "ead7268e-c4a2-41d1-a900-3aae7e924535"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:32.914100 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.914074 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgmv8\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-kube-api-access-dgmv8\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:32.914100 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.914097 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ead7268e-c4a2-41d1-a900-3aae7e924535-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:32.914259 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.914108 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:32.914259 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.914119 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:32.914259 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.914131 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:32.914259 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:32.914139 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ead7268e-c4a2-41d1-a900-3aae7e924535-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:33.179518 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.179489 2574 generic.go:356] "Generic (PLEG): container finished" podID="ead7268e-c4a2-41d1-a900-3aae7e924535" containerID="1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86" exitCode=0 Jul 06 01:55:33.179975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.179540 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" Jul 06 01:55:33.179975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.179583 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" event={"ID":"ead7268e-c4a2-41d1-a900-3aae7e924535","Type":"ContainerDied","Data":"1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86"} Jul 06 01:55:33.179975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.179642 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj" event={"ID":"ead7268e-c4a2-41d1-a900-3aae7e924535","Type":"ContainerDied","Data":"f692ae16502f6f92df7f1e5aebbe3026c53b32c2e983c9552fa58f2ca3db297d"} Jul 06 01:55:33.179975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.179670 2574 scope.go:117] "RemoveContainer" containerID="1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86" Jul 06 01:55:33.189932 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.189916 2574 scope.go:117] "RemoveContainer" containerID="1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86" Jul 06 01:55:33.190171 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:55:33.190151 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86\": container with ID starting with 1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86 not found: ID does not exist" containerID="1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86" Jul 06 01:55:33.190228 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.190185 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86"} err="failed to get container status \"1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86\": rpc error: code = NotFound desc = could not find container \"1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86\": container with ID starting with 1141cfd6bfb5c90a83ca5b0eda850ae15dd01dbd588aa3965bfc056ae716dc86 not found: ID does not exist" Jul 06 01:55:33.201655 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.201626 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj"] Jul 06 01:55:33.207317 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.207299 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a25d8b7a-openshift-default-84445dfb8c-c5pcj"] Jul 06 01:55:33.654175 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.654144 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7"] Jul 06 01:55:33.654574 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.654561 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ead7268e-c4a2-41d1-a900-3aae7e924535" containerName="istio-proxy" Jul 06 01:55:33.654628 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.654575 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ead7268e-c4a2-41d1-a900-3aae7e924535" containerName="istio-proxy" Jul 06 01:55:33.654673 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.654661 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ead7268e-c4a2-41d1-a900-3aae7e924535" containerName="istio-proxy" Jul 06 01:55:33.659459 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.659436 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.661746 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.661724 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2cf0b5a9-openshift-default-dockercfg-ncp9t\"" Jul 06 01:55:33.669210 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.669187 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7"] Jul 06 01:55:33.823693 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.823658 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-credential-socket\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.823693 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.823696 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-envoy\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.823971 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.823715 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-data\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.823971 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.823734 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-socket\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.823971 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.823788 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istiod-ca-cert\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.823971 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.823935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-certs\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.823971 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.823961 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-podinfo\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.824207 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.823989 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-token\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.824207 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.824054 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tmph\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-kube-api-access-5tmph\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925261 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925157 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-certs\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925261 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-podinfo\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925261 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925245 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-token\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925283 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tmph\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-kube-api-access-5tmph\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-credential-socket\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925379 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-envoy\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925402 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-data\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-socket\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925543 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925450 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istiod-ca-cert\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925910 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925601 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-certs\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925910 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925732 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-credential-socket\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925910 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925765 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-data\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.925910 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.925799 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-socket\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.926204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.926179 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istiod-ca-cert\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.927752 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.927729 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-envoy\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.927921 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.927905 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-podinfo\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.932992 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.932971 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-token\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.933078 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.933009 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tmph\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-kube-api-access-5tmph\") pod \"e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:33.972232 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:33.972202 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:34.115422 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:34.115397 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7"] Jul 06 01:55:34.116903 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:55:34.116851 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf86753d_22c0_4c66_9c91_2f3a7cb66554.slice/crio-76f3d616de8bbd3e2bfbb97e3d284c343c427cbbe82c98c390a9b96cc8ce91ea WatchSource:0}: Error finding container 76f3d616de8bbd3e2bfbb97e3d284c343c427cbbe82c98c390a9b96cc8ce91ea: Status 404 returned error can't find the container with id 76f3d616de8bbd3e2bfbb97e3d284c343c427cbbe82c98c390a9b96cc8ce91ea Jul 06 01:55:34.119282 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:34.119246 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:34.119390 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:34.119326 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:34.119390 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:34.119370 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:55:34.188644 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:34.188619 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" event={"ID":"bf86753d-22c0-4c66-9c91-2f3a7cb66554","Type":"ContainerStarted","Data":"76f3d616de8bbd3e2bfbb97e3d284c343c427cbbe82c98c390a9b96cc8ce91ea"} Jul 06 01:55:34.383178 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:34.383148 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ead7268e-c4a2-41d1-a900-3aae7e924535" path="/var/lib/kubelet/pods/ead7268e-c4a2-41d1-a900-3aae7e924535/volumes" Jul 06 01:55:35.196083 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:35.196053 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" event={"ID":"bf86753d-22c0-4c66-9c91-2f3a7cb66554","Type":"ContainerStarted","Data":"6bde999941cf770f70ddaea297e6a210e89a3ed6f20b471f996b8e4085bd4a76"} Jul 06 01:55:35.217175 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:35.217131 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" podStartSLOduration=2.217115743 podStartE2EDuration="2.217115743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:55:35.214318384 +0000 UTC m=+1733.349608885" watchObservedRunningTime="2026-07-06 01:55:35.217115743 +0000 UTC m=+1733.352406247" Jul 06 01:55:35.972908 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:35.972872 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:35.977474 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:35.977452 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:36.202886 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:36.202825 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:36.203793 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:36.203775 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:39.537040 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:39.537006 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7"] Jul 06 01:55:39.537498 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:39.537169 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" podUID="bf86753d-22c0-4c66-9c91-2f3a7cb66554" containerName="istio-proxy" containerID="cri-o://6bde999941cf770f70ddaea297e6a210e89a3ed6f20b471f996b8e4085bd4a76" gracePeriod=30 Jul 06 01:55:45.258345 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.258311 2574 generic.go:356] "Generic (PLEG): container finished" podID="bf86753d-22c0-4c66-9c91-2f3a7cb66554" containerID="6bde999941cf770f70ddaea297e6a210e89a3ed6f20b471f996b8e4085bd4a76" exitCode=0 Jul 06 01:55:45.258707 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.258354 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" event={"ID":"bf86753d-22c0-4c66-9c91-2f3a7cb66554","Type":"ContainerDied","Data":"6bde999941cf770f70ddaea297e6a210e89a3ed6f20b471f996b8e4085bd4a76"} Jul 06 01:55:45.394369 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.394344 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:45.439374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439305 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-data\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439338 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tmph\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-kube-api-access-5tmph\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439374 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439363 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-socket\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439380 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-token\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439436 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-credential-socket\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439472 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-certs\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439509 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istiod-ca-cert\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439542 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-podinfo\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439629 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-envoy\") pod \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\" (UID: \"bf86753d-22c0-4c66-9c91-2f3a7cb66554\") " Jul 06 01:55:45.439972 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439707 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:45.439972 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439846 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:45.439972 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.439716 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:45.440246 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.440222 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:55:45.440365 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.440318 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-data" (OuterVolumeSpecName: "istio-data") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:45.440561 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.440540 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:45.440801 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.440566 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:45.440801 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.440582 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:45.440801 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.440597 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:45.440801 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.440608 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:45.442164 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.442137 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-token" (OuterVolumeSpecName: "istio-token") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:55:45.442285 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.442253 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-kube-api-access-5tmph" (OuterVolumeSpecName: "kube-api-access-5tmph") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "kube-api-access-5tmph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:55:45.442618 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.442591 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:55:45.442701 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.442641 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bf86753d-22c0-4c66-9c91-2f3a7cb66554" (UID: "bf86753d-22c0-4c66-9c91-2f3a7cb66554"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 01:55:45.541417 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.541386 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tmph\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-kube-api-access-5tmph\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:45.541417 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.541418 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:45.541417 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.541429 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:45.541754 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:45.541441 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf86753d-22c0-4c66-9c91-2f3a7cb66554-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:55:46.265489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:46.265461 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" Jul 06 01:55:46.265489 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:46.265486 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7" event={"ID":"bf86753d-22c0-4c66-9c91-2f3a7cb66554","Type":"ContainerDied","Data":"76f3d616de8bbd3e2bfbb97e3d284c343c427cbbe82c98c390a9b96cc8ce91ea"} Jul 06 01:55:46.266053 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:46.265545 2574 scope.go:117] "RemoveContainer" containerID="6bde999941cf770f70ddaea297e6a210e89a3ed6f20b471f996b8e4085bd4a76" Jul 06 01:55:46.286716 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:46.286693 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7"] Jul 06 01:55:46.291052 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:46.291028 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2cf0b5a9-openshift-default-f48b9449f-q2pp7"] Jul 06 01:55:46.383069 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:55:46.383042 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf86753d-22c0-4c66-9c91-2f3a7cb66554" path="/var/lib/kubelet/pods/bf86753d-22c0-4c66-9c91-2f3a7cb66554/volumes" Jul 06 01:56:42.455471 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:56:42.455445 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:56:42.462280 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:56:42.462252 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:56:42.462955 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:56:42.462936 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 01:56:42.469556 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:56:42.469538 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 01:57:21.965661 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:21.965626 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm"] Jul 06 01:57:21.966142 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:21.966079 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf86753d-22c0-4c66-9c91-2f3a7cb66554" containerName="istio-proxy" Jul 06 01:57:21.966142 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:21.966092 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf86753d-22c0-4c66-9c91-2f3a7cb66554" containerName="istio-proxy" Jul 06 01:57:21.966215 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:21.966172 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf86753d-22c0-4c66-9c91-2f3a7cb66554" containerName="istio-proxy" Jul 06 01:57:21.969296 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:21.969279 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:21.972618 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:21.972596 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5a7d0b17-openshift-default-dockercfg-w4vqh\"" Jul 06 01:57:22.042067 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.042034 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm"] Jul 06 01:57:22.060038 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.059996 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-token\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.060239 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.060081 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istiod-ca-cert\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.060239 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.060132 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gqmh\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-kube-api-access-7gqmh\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.060239 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.060178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-data\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.060239 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.060213 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-certs\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.060467 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.060320 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-podinfo\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.060467 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.060373 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-socket\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.060575 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.060507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-envoy\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.060575 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.060537 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-credential-socket\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.161579 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161552 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istiod-ca-cert\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.161735 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161590 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gqmh\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-kube-api-access-7gqmh\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.161735 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161616 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-data\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.161735 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161640 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-certs\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.161735 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161666 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-podinfo\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.161735 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161689 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-socket\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.162054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161760 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-envoy\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.162054 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161843 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-credential-socket\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.162167 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.161899 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-token\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.162167 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.162118 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-certs\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.162278 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.162178 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-data\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.162278 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.162214 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-socket\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.162376 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.162310 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-credential-socket\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.162507 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.162482 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istiod-ca-cert\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.164245 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.164219 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-envoy\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.164716 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.164694 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-podinfo\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.172000 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.171974 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gqmh\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-kube-api-access-7gqmh\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.172083 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.172003 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-token\") pod \"e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.279666 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.279594 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:22.436650 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.436624 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm"] Jul 06 01:57:22.438211 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:57:22.438180 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e77e70_4d3a_44b7_9b02_dbc1ce071b57.slice/crio-290609ef4a3455a3dc1b6ab1fcece954a188d522be536a21ab1cc483eb6bf4d9 WatchSource:0}: Error finding container 290609ef4a3455a3dc1b6ab1fcece954a188d522be536a21ab1cc483eb6bf4d9: Status 404 returned error can't find the container with id 290609ef4a3455a3dc1b6ab1fcece954a188d522be536a21ab1cc483eb6bf4d9 Jul 06 01:57:22.440407 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.440367 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:57:22.440488 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.440450 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:57:22.440531 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.440491 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:57:22.810959 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.810918 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" event={"ID":"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57","Type":"ContainerStarted","Data":"3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8"} Jul 06 01:57:22.811160 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.810984 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" event={"ID":"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57","Type":"ContainerStarted","Data":"290609ef4a3455a3dc1b6ab1fcece954a188d522be536a21ab1cc483eb6bf4d9"} Jul 06 01:57:22.840016 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:22.839969 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" podStartSLOduration=1.839954444 podStartE2EDuration="1.839954444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:57:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:57:22.836851558 +0000 UTC m=+1840.972142062" watchObservedRunningTime="2026-07-06 01:57:22.839954444 +0000 UTC m=+1840.975244947" Jul 06 01:57:23.280585 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:23.280554 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:23.285397 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:23.285378 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:23.816168 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:23.816136 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:23.817045 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:23.817026 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:37.918366 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:37.918333 2574 generic.go:356] "Generic (PLEG): container finished" podID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerID="3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8" exitCode=0 Jul 06 01:57:37.918814 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:37.918420 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" event={"ID":"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57","Type":"ContainerDied","Data":"3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8"} Jul 06 01:57:37.918944 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:37.918930 2574 scope.go:117] "RemoveContainer" containerID="3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8" Jul 06 01:57:37.919971 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:37.919937 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:57:37.920099 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:37.920008 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:57:37.920099 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:37.920041 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:57:38.816456 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:38.816414 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:38.926224 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:38.926181 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" event={"ID":"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57","Type":"ContainerStarted","Data":"eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91"} Jul 06 01:57:39.281630 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:39.281586 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:39.281882 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:39.281723 2574 patch_prober.go:28] interesting pod/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 06 01:57:39.281882 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:39.281766 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 06 01:57:40.280581 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:40.280536 2574 patch_prober.go:28] interesting pod/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 06 01:57:40.281091 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:40.280606 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 06 01:57:41.280676 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:41.280636 2574 patch_prober.go:28] interesting pod/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 06 01:57:41.281178 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:41.280700 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 06 01:57:42.280124 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:42.280089 2574 patch_prober.go:28] interesting pod/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 06 01:57:42.280412 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:42.280382 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 06 01:57:43.284649 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:43.284617 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:43.285078 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:43.284899 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:43.285522 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:43.285502 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:43.590992 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:43.590961 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm"] Jul 06 01:57:43.954454 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:43.954348 2574 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" secret="" err="secret \"e2e-mt-5a7d0b17-openshift-default-dockercfg-w4vqh\" not found" Jul 06 01:57:44.959878 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:44.959794 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" containerID="cri-o://eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91" gracePeriod=30 Jul 06 01:57:50.218104 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.218051 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:50.252637 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252617 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-data\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.252740 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252710 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-envoy\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.252740 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252728 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-credential-socket\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.252828 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252751 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-certs\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.252828 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252769 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istiod-ca-cert\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.252828 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252785 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gqmh\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-kube-api-access-7gqmh\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.252828 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252800 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-socket\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.252828 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252826 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-token\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.253096 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252909 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-podinfo\") pod \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\" (UID: \"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57\") " Jul 06 01:57:50.253096 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.252995 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:57:50.253204 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.253183 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:57:50.253349 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.253301 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.253349 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.253327 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.253524 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.253451 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:57:50.253584 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.253536 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 01:57:50.253633 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.253584 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-data" (OuterVolumeSpecName: "istio-data") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:57:50.254931 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.254912 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 01:57:50.255378 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.255351 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-token" (OuterVolumeSpecName: "istio-token") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:57:50.255567 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.255537 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 01:57:50.255567 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.255556 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-kube-api-access-7gqmh" (OuterVolumeSpecName: "kube-api-access-7gqmh") pod "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" (UID: "d7e77e70-4d3a-44b7-9b02-dbc1ce071b57"). InnerVolumeSpecName "kube-api-access-7gqmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 01:57:50.353847 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.353814 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.353847 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.353841 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.353847 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.353849 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gqmh\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-kube-api-access-7gqmh\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.353847 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.353870 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.354119 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.353879 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.354119 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.353888 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.354119 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.353896 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 01:57:50.996036 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.996005 2574 generic.go:356] "Generic (PLEG): container finished" podID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerID="eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91" exitCode=0 Jul 06 01:57:50.996224 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.996051 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" event={"ID":"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57","Type":"ContainerDied","Data":"eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91"} Jul 06 01:57:50.996224 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.996065 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" Jul 06 01:57:50.996224 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.996095 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm" event={"ID":"d7e77e70-4d3a-44b7-9b02-dbc1ce071b57","Type":"ContainerDied","Data":"290609ef4a3455a3dc1b6ab1fcece954a188d522be536a21ab1cc483eb6bf4d9"} Jul 06 01:57:50.996224 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:50.996116 2574 scope.go:117] "RemoveContainer" containerID="eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91" Jul 06 01:57:51.008009 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:51.007991 2574 scope.go:117] "RemoveContainer" containerID="3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8" Jul 06 01:57:51.014946 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:51.014924 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm"] Jul 06 01:57:51.017531 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:51.017433 2574 scope.go:117] "RemoveContainer" containerID="eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91" Jul 06 01:57:51.017767 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:57:51.017740 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91\": container with ID starting with eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91 not found: ID does not exist" containerID="eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91" Jul 06 01:57:51.017887 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:51.017777 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91"} err="failed to get container status \"eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91\": rpc error: code = NotFound desc = could not find container \"eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91\": container with ID starting with eee48d52a681b97c98ec61335e88e09dfbb69e4428a4fbd8811007e51d6dfb91 not found: ID does not exist" Jul 06 01:57:51.017887 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:51.017801 2574 scope.go:117] "RemoveContainer" containerID="3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8" Jul 06 01:57:51.018107 ip-10-0-133-193 kubenswrapper[2574]: E0706 01:57:51.018087 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8\": container with ID starting with 3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8 not found: ID does not exist" containerID="3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8" Jul 06 01:57:51.018149 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:51.018114 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8"} err="failed to get container status \"3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8\": rpc error: code = NotFound desc = could not find container \"3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8\": container with ID starting with 3aecfb771317465ab1596bd826b58f7786d6b87fe4bb4991d355de3bb1e8e8c8 not found: ID does not exist" Jul 06 01:57:51.019813 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:51.019794 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5a7d0b17-openshift-default-869f559f9f-6fcgm"] Jul 06 01:57:52.383153 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:57:52.383119 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" path="/var/lib/kubelet/pods/d7e77e70-4d3a-44b7-9b02-dbc1ce071b57/volumes" Jul 06 01:58:03.988416 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:03.988339 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn"] Jul 06 01:58:03.988837 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:03.988785 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" Jul 06 01:58:03.988837 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:03.988796 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" Jul 06 01:58:03.988837 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:03.988820 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" Jul 06 01:58:03.988837 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:03.988825 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" Jul 06 01:58:03.988991 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:03.988907 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" Jul 06 01:58:03.993672 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:03.993652 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:03.996389 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:03.996366 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-95397071-openshift-default-dockercfg-5fxzn\"" Jul 06 01:58:04.005596 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.005571 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn"] Jul 06 01:58:04.083035 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083001 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-token\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.083182 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083043 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-podinfo\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.083182 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083096 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-envoy\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.083182 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083142 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-data\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.083182 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083159 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba9783-2dee-4991-b9fb-a28c7163b409-istiod-ca-cert\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.083322 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083186 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn97r\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-kube-api-access-dn97r\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.083322 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-certs\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.083322 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-socket\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.083322 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.083295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-credential-socket\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.184692 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.184656 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-token\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.184884 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.184703 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-podinfo\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.184884 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.184729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-envoy\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.184975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.184916 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-data\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.184975 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.184950 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba9783-2dee-4991-b9fb-a28c7163b409-istiod-ca-cert\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185046 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.184992 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn97r\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-kube-api-access-dn97r\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185094 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.185041 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-certs\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185094 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.185072 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-socket\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185197 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.185119 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-credential-socket\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185337 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.185289 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-data\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185469 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.185426 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-certs\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185549 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.185483 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-credential-socket\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185549 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.185531 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-socket\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.185664 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.185639 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba9783-2dee-4991-b9fb-a28c7163b409-istiod-ca-cert\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.187759 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.187734 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-envoy\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.187969 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.187954 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-podinfo\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.192683 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.192663 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-token\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.192971 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.192952 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn97r\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-kube-api-access-dn97r\") pod \"e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.305002 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.304924 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:04.450979 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.450947 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn"] Jul 06 01:58:04.452246 ip-10-0-133-193 kubenswrapper[2574]: W0706 01:58:04.452220 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ba9783_2dee_4991_b9fb_a28c7163b409.slice/crio-ccd79027d696b78aa8fac8443e36672775f86c79fc4f04887ce46435391e3b28 WatchSource:0}: Error finding container ccd79027d696b78aa8fac8443e36672775f86c79fc4f04887ce46435391e3b28: Status 404 returned error can't find the container with id ccd79027d696b78aa8fac8443e36672775f86c79fc4f04887ce46435391e3b28 Jul 06 01:58:04.454421 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.454376 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:58:04.454525 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.454453 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:58:04.454525 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:04.454479 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:58:05.077424 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:05.077390 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" event={"ID":"a2ba9783-2dee-4991-b9fb-a28c7163b409","Type":"ContainerStarted","Data":"94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d"} Jul 06 01:58:05.077820 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:05.077438 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" event={"ID":"a2ba9783-2dee-4991-b9fb-a28c7163b409","Type":"ContainerStarted","Data":"ccd79027d696b78aa8fac8443e36672775f86c79fc4f04887ce46435391e3b28"} Jul 06 01:58:05.095586 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:05.095523 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" podStartSLOduration=2.095506807 podStartE2EDuration="2.095506807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 01:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 01:58:05.095063929 +0000 UTC m=+1883.230354437" watchObservedRunningTime="2026-07-06 01:58:05.095506807 +0000 UTC m=+1883.230797312" Jul 06 01:58:05.305116 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:05.305086 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:05.310231 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:05.310208 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:06.083565 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:06.083532 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:06.084559 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:06.084540 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:21.524631 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:21.523961 2574 patch_prober.go:28] interesting pod/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:39014->10.133.0.57:15021: read: connection reset by peer" start-of-body= Jul 06 01:58:21.524631 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:21.524021 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:39014->10.133.0.57:15021: read: connection reset by peer" Jul 06 01:58:22.177821 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:22.177784 2574 generic.go:356] "Generic (PLEG): container finished" podID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerID="94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d" exitCode=0 Jul 06 01:58:22.178024 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:22.177887 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" event={"ID":"a2ba9783-2dee-4991-b9fb-a28c7163b409","Type":"ContainerDied","Data":"94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d"} Jul 06 01:58:22.178313 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:22.178299 2574 scope.go:117] "RemoveContainer" containerID="94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d" Jul 06 01:58:22.179301 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:22.179271 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:58:22.179377 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:22.179338 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:58:22.179377 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:22.179371 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 01:58:23.185453 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:23.185421 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" event={"ID":"a2ba9783-2dee-4991-b9fb-a28c7163b409","Type":"ContainerStarted","Data":"9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44"} Jul 06 01:58:23.305183 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:23.305147 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:23.305411 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:23.305389 2574 patch_prober.go:28] interesting pod/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 06 01:58:23.305502 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:23.305440 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 06 01:58:24.306274 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:24.306244 2574 patch_prober.go:28] interesting pod/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 06 01:58:24.306653 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:24.306294 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 06 01:58:25.305947 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:25.305913 2574 patch_prober.go:28] interesting pod/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 06 01:58:25.306127 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:25.305968 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 06 01:58:26.306230 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:26.306198 2574 patch_prober.go:28] interesting pod/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 06 01:58:26.306617 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:26.306250 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 06 01:58:27.305383 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:27.305352 2574 patch_prober.go:28] interesting pod/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 06 01:58:27.305546 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:27.305403 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 06 01:58:28.309048 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:28.309018 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:28.309410 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:28.309352 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 01:58:28.309939 ip-10-0-133-193 kubenswrapper[2574]: I0706 01:58:28.309924 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 02:00:00.141763 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.141680 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29721720-5v9h2"] Jul 06 02:00:00.142280 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.142189 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e77e70-4d3a-44b7-9b02-dbc1ce071b57" containerName="istio-proxy" Jul 06 02:00:00.145236 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.145220 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" Jul 06 02:00:00.157891 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.157849 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29721720-5v9h2"] Jul 06 02:00:00.285249 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.285211 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9grh5\" (UniqueName: \"kubernetes.io/projected/be3e159d-d3c4-4954-bb8b-0ee8fa05719e-kube-api-access-9grh5\") pod \"maas-api-key-cleanup-29721720-5v9h2\" (UID: \"be3e159d-d3c4-4954-bb8b-0ee8fa05719e\") " pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" Jul 06 02:00:00.386210 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.386177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9grh5\" (UniqueName: \"kubernetes.io/projected/be3e159d-d3c4-4954-bb8b-0ee8fa05719e-kube-api-access-9grh5\") pod \"maas-api-key-cleanup-29721720-5v9h2\" (UID: \"be3e159d-d3c4-4954-bb8b-0ee8fa05719e\") " pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" Jul 06 02:00:00.394698 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.394630 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9grh5\" (UniqueName: \"kubernetes.io/projected/be3e159d-d3c4-4954-bb8b-0ee8fa05719e-kube-api-access-9grh5\") pod \"maas-api-key-cleanup-29721720-5v9h2\" (UID: \"be3e159d-d3c4-4954-bb8b-0ee8fa05719e\") " pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" Jul 06 02:00:00.455723 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.455686 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" Jul 06 02:00:00.598113 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.598090 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29721720-5v9h2"] Jul 06 02:00:00.600352 ip-10-0-133-193 kubenswrapper[2574]: W0706 02:00:00.600321 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3e159d_d3c4_4954_bb8b_0ee8fa05719e.slice/crio-ebb5e314fd16e1bdc580db8856aa8cabf5322f55bb74aefa747c7f50a2dc4de3 WatchSource:0}: Error finding container ebb5e314fd16e1bdc580db8856aa8cabf5322f55bb74aefa747c7f50a2dc4de3: Status 404 returned error can't find the container with id ebb5e314fd16e1bdc580db8856aa8cabf5322f55bb74aefa747c7f50a2dc4de3 Jul 06 02:00:00.602086 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.602071 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 02:00:00.756460 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:00.756370 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" event={"ID":"be3e159d-d3c4-4954-bb8b-0ee8fa05719e","Type":"ContainerStarted","Data":"ebb5e314fd16e1bdc580db8856aa8cabf5322f55bb74aefa747c7f50a2dc4de3"} Jul 06 02:00:14.848930 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:14.848891 2574 generic.go:356] "Generic (PLEG): container finished" podID="be3e159d-d3c4-4954-bb8b-0ee8fa05719e" containerID="703e01a0942ba85fcf489b406c1d932fc50e4be07a23532cf15c444eca261de9" exitCode=0 Jul 06 02:00:14.849320 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:14.848974 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" event={"ID":"be3e159d-d3c4-4954-bb8b-0ee8fa05719e","Type":"ContainerDied","Data":"703e01a0942ba85fcf489b406c1d932fc50e4be07a23532cf15c444eca261de9"} Jul 06 02:00:15.999277 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:15.999254 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" Jul 06 02:00:16.052217 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:16.052179 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9grh5\" (UniqueName: \"kubernetes.io/projected/be3e159d-d3c4-4954-bb8b-0ee8fa05719e-kube-api-access-9grh5\") pod \"be3e159d-d3c4-4954-bb8b-0ee8fa05719e\" (UID: \"be3e159d-d3c4-4954-bb8b-0ee8fa05719e\") " Jul 06 02:00:16.054843 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:16.054801 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be3e159d-d3c4-4954-bb8b-0ee8fa05719e-kube-api-access-9grh5" (OuterVolumeSpecName: "kube-api-access-9grh5") pod "be3e159d-d3c4-4954-bb8b-0ee8fa05719e" (UID: "be3e159d-d3c4-4954-bb8b-0ee8fa05719e"). InnerVolumeSpecName "kube-api-access-9grh5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:00:16.153654 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:16.153554 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9grh5\" (UniqueName: \"kubernetes.io/projected/be3e159d-d3c4-4954-bb8b-0ee8fa05719e-kube-api-access-9grh5\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:00:16.862147 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:16.862119 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" Jul 06 02:00:16.862335 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:16.862147 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29721720-5v9h2" event={"ID":"be3e159d-d3c4-4954-bb8b-0ee8fa05719e","Type":"ContainerDied","Data":"ebb5e314fd16e1bdc580db8856aa8cabf5322f55bb74aefa747c7f50a2dc4de3"} Jul 06 02:00:16.862335 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:00:16.862195 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ebb5e314fd16e1bdc580db8856aa8cabf5322f55bb74aefa747c7f50a2dc4de3" Jul 06 02:01:22.961500 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:22.961465 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn"] Jul 06 02:01:22.962120 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:22.961673 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" containerID="cri-o://9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44" gracePeriod=30 Jul 06 02:01:28.228963 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.228899 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 02:01:28.290321 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.290292 2574 generic.go:356] "Generic (PLEG): container finished" podID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerID="9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44" exitCode=0 Jul 06 02:01:28.290505 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.290342 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" Jul 06 02:01:28.290505 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.290383 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" event={"ID":"a2ba9783-2dee-4991-b9fb-a28c7163b409","Type":"ContainerDied","Data":"9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44"} Jul 06 02:01:28.290505 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.290448 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn" event={"ID":"a2ba9783-2dee-4991-b9fb-a28c7163b409","Type":"ContainerDied","Data":"ccd79027d696b78aa8fac8443e36672775f86c79fc4f04887ce46435391e3b28"} Jul 06 02:01:28.290505 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.290474 2574 scope.go:117] "RemoveContainer" containerID="9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44" Jul 06 02:01:28.300120 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.300101 2574 scope.go:117] "RemoveContainer" containerID="94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d" Jul 06 02:01:28.304414 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304392 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-certs\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.304537 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304431 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-podinfo\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.304537 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304467 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-credential-socket\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.304650 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304555 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-socket\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.304650 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304634 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba9783-2dee-4991-b9fb-a28c7163b409-istiod-ca-cert\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.304747 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304670 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:01:28.304747 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304665 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-token\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.304846 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304747 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-envoy\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.304846 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304773 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn97r\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-kube-api-access-dn97r\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.304846 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.304819 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-data\") pod \"a2ba9783-2dee-4991-b9fb-a28c7163b409\" (UID: \"a2ba9783-2dee-4991-b9fb-a28c7163b409\") " Jul 06 02:01:28.305354 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.305034 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:01:28.305354 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.305109 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2ba9783-2dee-4991-b9fb-a28c7163b409-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:01:28.305354 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.305224 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:01:28.305354 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.305352 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.305620 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.305369 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.305620 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.305382 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.305620 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.305395 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2ba9783-2dee-4991-b9fb-a28c7163b409-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.305800 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.305779 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:01:28.306999 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.306973 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:01:28.307472 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.307444 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 02:01:28.307571 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.307444 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:01:28.307571 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.307508 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-kube-api-access-dn97r" (OuterVolumeSpecName: "kube-api-access-dn97r") pod "a2ba9783-2dee-4991-b9fb-a28c7163b409" (UID: "a2ba9783-2dee-4991-b9fb-a28c7163b409"). InnerVolumeSpecName "kube-api-access-dn97r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:01:28.310785 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.310767 2574 scope.go:117] "RemoveContainer" containerID="9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44" Jul 06 02:01:28.311095 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:01:28.311073 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44\": container with ID starting with 9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44 not found: ID does not exist" containerID="9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44" Jul 06 02:01:28.311164 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.311101 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44"} err="failed to get container status \"9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44\": rpc error: code = NotFound desc = could not find container \"9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44\": container with ID starting with 9e0c9432fd2d50889d30409c4c4e9ec076af9da28c4b6d85307625a3d2afce44 not found: ID does not exist" Jul 06 02:01:28.311164 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.311121 2574 scope.go:117] "RemoveContainer" containerID="94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d" Jul 06 02:01:28.311379 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:01:28.311360 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d\": container with ID starting with 94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d not found: ID does not exist" containerID="94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d" Jul 06 02:01:28.311422 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.311384 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d"} err="failed to get container status \"94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d\": rpc error: code = NotFound desc = could not find container \"94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d\": container with ID starting with 94c9577bc91cb5f341a3c1b83d00ade5cd5eedbc720506cad4ab9132f4ab3c8d not found: ID does not exist" Jul 06 02:01:28.406160 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.406125 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.406160 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.406155 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.406362 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.406168 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn97r\" (UniqueName: \"kubernetes.io/projected/a2ba9783-2dee-4991-b9fb-a28c7163b409-kube-api-access-dn97r\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.406362 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.406185 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.406362 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.406198 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2ba9783-2dee-4991-b9fb-a28c7163b409-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:01:28.607975 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.607944 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn"] Jul 06 02:01:28.611800 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:28.611771 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-95397071-openshift-default-955f4d4bc-ksfqn"] Jul 06 02:01:30.382272 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:30.382238 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" path="/var/lib/kubelet/pods/a2ba9783-2dee-4991-b9fb-a28c7163b409/volumes" Jul 06 02:01:42.501544 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:42.501509 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 02:01:42.507896 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:42.507846 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 02:01:42.508280 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:42.508264 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 02:01:42.514761 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:01:42.514742 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 02:02:17.501125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.501086 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk"] Jul 06 02:02:17.502031 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.501999 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" Jul 06 02:02:17.502031 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.502029 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" Jul 06 02:02:17.502250 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.502058 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" Jul 06 02:02:17.502250 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.502068 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" Jul 06 02:02:17.502250 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.502106 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be3e159d-d3c4-4954-bb8b-0ee8fa05719e" containerName="cleanup" Jul 06 02:02:17.502250 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.502115 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="be3e159d-d3c4-4954-bb8b-0ee8fa05719e" containerName="cleanup" Jul 06 02:02:17.502535 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.502267 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" Jul 06 02:02:17.502535 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.502285 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2ba9783-2dee-4991-b9fb-a28c7163b409" containerName="istio-proxy" Jul 06 02:02:17.502535 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.502294 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="be3e159d-d3c4-4954-bb8b-0ee8fa05719e" containerName="cleanup" Jul 06 02:02:17.505417 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.505399 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.507638 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.507619 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f285a0-openshift-default-dockercfg-l5b4j\"" Jul 06 02:02:17.517805 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.517769 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk"] Jul 06 02:02:17.599188 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-socket\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.599372 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599196 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-envoy\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.599372 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599284 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istiod-ca-cert\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.599372 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599341 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-data\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.599537 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599491 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k29xl\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-kube-api-access-k29xl\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.599597 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599537 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-credential-socket\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.599597 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-certs\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.599702 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599621 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-token\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.599702 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.599684 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-podinfo\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701266 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701228 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k29xl\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-kube-api-access-k29xl\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701266 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701270 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-credential-socket\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701558 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701287 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-certs\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701558 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701306 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-token\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701558 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-podinfo\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701558 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701402 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-socket\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701777 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701556 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-envoy\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701777 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701635 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istiod-ca-cert\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701777 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701701 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-data\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.701777 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701760 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-credential-socket\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.702021 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701797 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-certs\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.702021 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.701935 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-socket\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.702123 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.702071 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-data\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.702302 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.702249 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istiod-ca-cert\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.704028 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.704006 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-envoy\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.704231 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.704214 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-podinfo\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.708348 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.708326 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-token\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.708626 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.708603 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k29xl\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-kube-api-access-k29xl\") pod \"e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.817062 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.816967 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:17.965879 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.965833 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk"] Jul 06 02:02:17.967744 ip-10-0-133-193 kubenswrapper[2574]: W0706 02:02:17.967708 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11e9bfd3_f703_4463_ba2e_b5ed4526a990.slice/crio-ecfd782a5b58e32178756f1fe6d15881fcb1a1ece3e125a5ffb3d7e5678ff220 WatchSource:0}: Error finding container ecfd782a5b58e32178756f1fe6d15881fcb1a1ece3e125a5ffb3d7e5678ff220: Status 404 returned error can't find the container with id ecfd782a5b58e32178756f1fe6d15881fcb1a1ece3e125a5ffb3d7e5678ff220 Jul 06 02:02:17.970314 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.970277 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:17.970411 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.970355 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:17.970411 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:17.970384 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:18.584663 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:18.584625 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" event={"ID":"11e9bfd3-f703-4463-ba2e-b5ed4526a990","Type":"ContainerStarted","Data":"371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d"} Jul 06 02:02:18.585096 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:18.584687 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" event={"ID":"11e9bfd3-f703-4463-ba2e-b5ed4526a990","Type":"ContainerStarted","Data":"ecfd782a5b58e32178756f1fe6d15881fcb1a1ece3e125a5ffb3d7e5678ff220"} Jul 06 02:02:18.606930 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:18.606821 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" podStartSLOduration=1.606799851 podStartE2EDuration="1.606799851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:02:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:02:18.606162477 +0000 UTC m=+2136.741452982" watchObservedRunningTime="2026-07-06 02:02:18.606799851 +0000 UTC m=+2136.742090354" Jul 06 02:02:18.817960 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:18.817924 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:19.822504 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:19.822476 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:20.597400 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:20.597366 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:20.598436 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:20.598414 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:25.790732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.790676 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr"] Jul 06 02:02:25.795323 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.795298 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.797353 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.797326 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-782aa6-openshift-default-dockercfg-zk5w7\"" Jul 06 02:02:25.804992 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.804968 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr"] Jul 06 02:02:25.884660 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.884615 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istiod-ca-cert\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.884944 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.884673 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-token\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.884944 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.884701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-certs\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.884944 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.884756 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-credential-socket\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.884944 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.884829 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-data\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.885153 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.884957 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn866\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-kube-api-access-fn866\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.885153 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.885002 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-envoy\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.885153 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.885077 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-podinfo\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.885153 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.885141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-socket\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.986673 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986600 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istiod-ca-cert\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.986673 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986653 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-token\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987010 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986682 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-certs\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987010 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986742 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-credential-socket\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987010 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986790 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-data\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987010 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986838 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn866\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-kube-api-access-fn866\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987010 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-envoy\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987010 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986929 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-podinfo\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987010 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.986968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-socket\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987377 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.987177 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-certs\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987377 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.987217 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-credential-socket\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987377 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.987276 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-data\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987541 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.987380 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-socket\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.987541 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.987426 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istiod-ca-cert\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.989385 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.989361 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-envoy\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.989620 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.989592 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-podinfo\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.993930 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.993903 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-token\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:25.994133 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:25.994113 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn866\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-kube-api-access-fn866\") pod \"e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:26.109126 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:26.109086 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:26.257637 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:26.257610 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr"] Jul 06 02:02:26.260270 ip-10-0-133-193 kubenswrapper[2574]: W0706 02:02:26.260244 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb95a891_db20_40ac_a3fb_51e8dc3f97f5.slice/crio-1e03bba5080b15caeebc0f94a2359edf6f3cf692adcd2a04dcf2516dc4e215b1 WatchSource:0}: Error finding container 1e03bba5080b15caeebc0f94a2359edf6f3cf692adcd2a04dcf2516dc4e215b1: Status 404 returned error can't find the container with id 1e03bba5080b15caeebc0f94a2359edf6f3cf692adcd2a04dcf2516dc4e215b1 Jul 06 02:02:26.263059 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:26.263026 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:26.263139 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:26.263094 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:26.263139 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:26.263121 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:26.634575 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:26.634467 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" event={"ID":"bb95a891-db20-40ac-a3fb-51e8dc3f97f5","Type":"ContainerStarted","Data":"a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463"} Jul 06 02:02:26.634575 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:26.634535 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" event={"ID":"bb95a891-db20-40ac-a3fb-51e8dc3f97f5","Type":"ContainerStarted","Data":"1e03bba5080b15caeebc0f94a2359edf6f3cf692adcd2a04dcf2516dc4e215b1"} Jul 06 02:02:26.653450 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:26.653393 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" podStartSLOduration=1.6533775579999999 podStartE2EDuration="1.653377558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:02:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:02:26.652152512 +0000 UTC m=+2144.787443016" watchObservedRunningTime="2026-07-06 02:02:26.653377558 +0000 UTC m=+2144.788668061" Jul 06 02:02:27.110053 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:27.110006 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:27.115404 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:27.115380 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:27.642227 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:27.642195 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:27.643202 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:27.643183 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:36.638047 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:36.638007 2574 patch_prober.go:28] interesting pod/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 02:02:36.638454 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:36.638071 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 02:02:37.726039 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:37.726002 2574 generic.go:356] "Generic (PLEG): container finished" podID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerID="371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d" exitCode=0 Jul 06 02:02:37.726485 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:37.726054 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" event={"ID":"11e9bfd3-f703-4463-ba2e-b5ed4526a990","Type":"ContainerDied","Data":"371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d"} Jul 06 02:02:37.726485 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:37.726473 2574 scope.go:117] "RemoveContainer" containerID="371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d" Jul 06 02:02:37.727598 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:37.727562 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:37.727695 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:37.727648 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:37.727695 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:37.727681 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:38.738984 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:38.738932 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" event={"ID":"11e9bfd3-f703-4463-ba2e-b5ed4526a990","Type":"ContainerStarted","Data":"1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06"} Jul 06 02:02:38.817367 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:38.817327 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:38.817544 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:38.817435 2574 patch_prober.go:28] interesting pod/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 06 02:02:38.817544 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:38.817474 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 06 02:02:39.818361 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:39.818324 2574 patch_prober.go:28] interesting pod/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 06 02:02:39.818795 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:39.818383 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 06 02:02:40.818160 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:40.818119 2574 patch_prober.go:28] interesting pod/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 06 02:02:40.818353 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:40.818213 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 06 02:02:41.817982 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:41.817942 2574 patch_prober.go:28] interesting pod/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 06 02:02:41.818624 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:41.818018 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 06 02:02:42.818416 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:42.818357 2574 patch_prober.go:28] interesting pod/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 06 02:02:42.818957 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:42.818431 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 06 02:02:43.685065 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:43.685007 2574 patch_prober.go:28] interesting pod/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 02:02:43.685274 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:43.685096 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 02:02:43.821640 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:43.821610 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:43.822966 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:43.821910 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:43.822966 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:43.822598 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:02:44.780604 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:44.780573 2574 generic.go:356] "Generic (PLEG): container finished" podID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerID="a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463" exitCode=0 Jul 06 02:02:44.780789 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:44.780650 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" event={"ID":"bb95a891-db20-40ac-a3fb-51e8dc3f97f5","Type":"ContainerDied","Data":"a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463"} Jul 06 02:02:44.781193 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:44.781178 2574 scope.go:117] "RemoveContainer" containerID="a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463" Jul 06 02:02:44.782159 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:44.782133 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:44.782224 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:44.782204 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:44.782266 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:44.782238 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:02:45.790435 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:45.790394 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" event={"ID":"bb95a891-db20-40ac-a3fb-51e8dc3f97f5","Type":"ContainerStarted","Data":"9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb"} Jul 06 02:02:46.110290 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:46.110253 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:46.110484 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:46.110468 2574 patch_prober.go:28] interesting pod/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 06 02:02:46.110539 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:46.110516 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 06 02:02:47.110060 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:47.110017 2574 patch_prober.go:28] interesting pod/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 06 02:02:47.110505 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:47.110095 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 06 02:02:48.110002 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:48.109956 2574 patch_prober.go:28] interesting pod/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 06 02:02:48.110247 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:48.110043 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 06 02:02:49.110476 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:49.110440 2574 patch_prober.go:28] interesting pod/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 06 02:02:49.110891 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:49.110497 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 06 02:02:50.113781 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:50.113747 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:50.114296 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:50.114252 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:02:50.114957 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:02:50.114941 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:04:30.673314 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:30.673276 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk"] Jul 06 02:04:30.673765 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:30.673510 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" containerID="cri-o://1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06" gracePeriod=30 Jul 06 02:04:35.934088 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:35.934017 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:04:36.036220 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036184 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-podinfo\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036426 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036287 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k29xl\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-kube-api-access-k29xl\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036426 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036381 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-credential-socket\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036544 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036432 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-socket\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036544 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036485 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-envoy\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036544 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036507 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-certs\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036700 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036551 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-token\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036700 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036596 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-data\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036700 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036627 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istiod-ca-cert\") pod \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\" (UID: \"11e9bfd3-f703-4463-ba2e-b5ed4526a990\") " Jul 06 02:04:36.036700 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036667 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:36.036910 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.036812 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:36.037226 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.037152 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.037226 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.037178 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.037226 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.037147 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:04:36.037512 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.037486 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:36.037608 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.037589 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-data" (OuterVolumeSpecName: "istio-data") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:36.038966 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.038935 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-kube-api-access-k29xl" (OuterVolumeSpecName: "kube-api-access-k29xl") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "kube-api-access-k29xl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:04:36.039071 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.038958 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-token" (OuterVolumeSpecName: "istio-token") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:04:36.039071 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.038982 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:36.039071 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.039004 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11e9bfd3-f703-4463-ba2e-b5ed4526a990" (UID: "11e9bfd3-f703-4463-ba2e-b5ed4526a990"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 02:04:36.137974 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.137923 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k29xl\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-kube-api-access-k29xl\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.137974 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.137965 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.137974 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.137982 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.137974 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.137992 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.138394 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.138006 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.138394 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.138017 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.138394 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.138029 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11e9bfd3-f703-4463-ba2e-b5ed4526a990-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:36.462659 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.462626 2574 generic.go:356] "Generic (PLEG): container finished" podID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerID="1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06" exitCode=0 Jul 06 02:04:36.462879 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.462686 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" Jul 06 02:04:36.462879 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.462721 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" event={"ID":"11e9bfd3-f703-4463-ba2e-b5ed4526a990","Type":"ContainerDied","Data":"1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06"} Jul 06 02:04:36.462879 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.462780 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk" event={"ID":"11e9bfd3-f703-4463-ba2e-b5ed4526a990","Type":"ContainerDied","Data":"ecfd782a5b58e32178756f1fe6d15881fcb1a1ece3e125a5ffb3d7e5678ff220"} Jul 06 02:04:36.462879 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.462805 2574 scope.go:117] "RemoveContainer" containerID="1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06" Jul 06 02:04:36.472804 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.472787 2574 scope.go:117] "RemoveContainer" containerID="371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d" Jul 06 02:04:36.480603 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.480579 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk"] Jul 06 02:04:36.482667 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.482651 2574 scope.go:117] "RemoveContainer" containerID="1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06" Jul 06 02:04:36.482963 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:04:36.482934 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06\": container with ID starting with 1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06 not found: ID does not exist" containerID="1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06" Jul 06 02:04:36.483052 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.482973 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06"} err="failed to get container status \"1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06\": rpc error: code = NotFound desc = could not find container \"1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06\": container with ID starting with 1d681aa9978353f894d14c355788b5107d1c121edd019a0200ccd63e45406e06 not found: ID does not exist" Jul 06 02:04:36.483052 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.482994 2574 scope.go:117] "RemoveContainer" containerID="371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d" Jul 06 02:04:36.483326 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:04:36.483288 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d\": container with ID starting with 371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d not found: ID does not exist" containerID="371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d" Jul 06 02:04:36.483441 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.483335 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d"} err="failed to get container status \"371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d\": rpc error: code = NotFound desc = could not find container \"371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d\": container with ID starting with 371da1ccd7f16f2faf4d8b1c2a80e819778a4421130943d34c8ac1e06283806d not found: ID does not exist" Jul 06 02:04:36.486798 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:36.486779 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f285a0-openshift-default-bb86d4bfc-wzlgk"] Jul 06 02:04:37.702075 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:37.702039 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr"] Jul 06 02:04:37.702459 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:37.702294 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" containerID="cri-o://9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb" gracePeriod=30 Jul 06 02:04:38.382704 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.382673 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" path="/var/lib/kubelet/pods/11e9bfd3-f703-4463-ba2e-b5ed4526a990/volumes" Jul 06 02:04:38.586666 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.586575 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs"] Jul 06 02:04:38.587339 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.587310 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" Jul 06 02:04:38.587339 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.587337 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" Jul 06 02:04:38.587564 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.587350 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" Jul 06 02:04:38.587564 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.587433 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" Jul 06 02:04:38.587645 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.587580 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" Jul 06 02:04:38.590641 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.590621 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.593110 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.593085 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0cd54e-openshift-default-dockercfg-s8lkc\"" Jul 06 02:04:38.605381 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.605354 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs"] Jul 06 02:04:38.663488 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663405 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-token\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.663488 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663437 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6h7n\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-kube-api-access-q6h7n\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.663488 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663463 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-envoy\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.663762 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663492 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-socket\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.663762 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663531 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-data\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.663762 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663547 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-credential-socket\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.663762 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istiod-ca-cert\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.663762 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-certs\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.663762 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.663701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-podinfo\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.764596 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istiod-ca-cert\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764623 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-certs\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764656 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-podinfo\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764740 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-token\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764769 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6h7n\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-kube-api-access-q6h7n\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764804 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-envoy\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764842 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-socket\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764904 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-data\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765125 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.764930 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-credential-socket\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765727 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.765264 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-socket\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765727 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.765431 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istiod-ca-cert\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765727 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.765468 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-certs\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765727 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.765569 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-credential-socket\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.765727 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.765595 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-data\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.767432 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.767405 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-envoy\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.767612 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.767596 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-podinfo\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.772541 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.772517 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6h7n\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-kube-api-access-q6h7n\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.773133 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.773116 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-token\") pod \"e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:38.902028 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:38.901994 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:39.052092 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:39.052064 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs"] Jul 06 02:04:39.053899 ip-10-0-133-193 kubenswrapper[2574]: W0706 02:04:39.053849 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc9db81a_cc6b_4aa1_928f_4f25027ac0bb.slice/crio-06dc27d5cd3ac77b269c402506881137fcf388c7de3f4e2d667145035d6d2617 WatchSource:0}: Error finding container 06dc27d5cd3ac77b269c402506881137fcf388c7de3f4e2d667145035d6d2617: Status 404 returned error can't find the container with id 06dc27d5cd3ac77b269c402506881137fcf388c7de3f4e2d667145035d6d2617 Jul 06 02:04:39.056293 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:39.056249 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:39.056375 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:39.056345 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:39.056415 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:39.056394 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:39.487160 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:39.487124 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" event={"ID":"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb","Type":"ContainerStarted","Data":"dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68"} Jul 06 02:04:39.487389 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:39.487186 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" event={"ID":"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb","Type":"ContainerStarted","Data":"06dc27d5cd3ac77b269c402506881137fcf388c7de3f4e2d667145035d6d2617"} Jul 06 02:04:39.511661 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:39.511604 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" podStartSLOduration=1.511588631 podStartE2EDuration="1.511588631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:04:39.508965898 +0000 UTC m=+2277.644256402" watchObservedRunningTime="2026-07-06 02:04:39.511588631 +0000 UTC m=+2277.646879136" Jul 06 02:04:39.902997 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:39.902959 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:40.907904 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:40.907851 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:41.498670 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:41.498637 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:41.499661 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:41.499632 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:42.974979 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:42.974951 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:04:43.109302 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109265 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn866\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-kube-api-access-fn866\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109302 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109305 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-token\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109532 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109337 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istiod-ca-cert\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109532 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109382 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-data\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109532 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109408 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-envoy\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109532 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109500 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-podinfo\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109749 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109545 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-certs\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109749 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109572 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-socket\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109749 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109633 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-credential-socket\") pod \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\" (UID: \"bb95a891-db20-40ac-a3fb-51e8dc3f97f5\") " Jul 06 02:04:43.109955 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109802 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:43.109955 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.109852 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:04:43.110068 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.110025 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:43.110068 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.110047 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:43.110286 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.110266 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.110286 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.110288 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.110473 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.110301 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.110473 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.110312 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.110666 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.110616 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:43.112023 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.111997 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 02:04:43.112023 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.112005 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:04:43.112193 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.112022 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:04:43.112259 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.112238 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-kube-api-access-fn866" (OuterVolumeSpecName: "kube-api-access-fn866") pod "bb95a891-db20-40ac-a3fb-51e8dc3f97f5" (UID: "bb95a891-db20-40ac-a3fb-51e8dc3f97f5"). InnerVolumeSpecName "kube-api-access-fn866". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:04:43.211518 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.211427 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn866\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-kube-api-access-fn866\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.211518 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.211458 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.211518 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.211468 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.211518 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.211475 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.211518 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.211484 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb95a891-db20-40ac-a3fb-51e8dc3f97f5-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:04:43.511480 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.511387 2574 generic.go:356] "Generic (PLEG): container finished" podID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerID="9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb" exitCode=0 Jul 06 02:04:43.511480 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.511436 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" Jul 06 02:04:43.511480 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.511473 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" event={"ID":"bb95a891-db20-40ac-a3fb-51e8dc3f97f5","Type":"ContainerDied","Data":"9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb"} Jul 06 02:04:43.511757 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.511526 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr" event={"ID":"bb95a891-db20-40ac-a3fb-51e8dc3f97f5","Type":"ContainerDied","Data":"1e03bba5080b15caeebc0f94a2359edf6f3cf692adcd2a04dcf2516dc4e215b1"} Jul 06 02:04:43.511757 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.511550 2574 scope.go:117] "RemoveContainer" containerID="9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb" Jul 06 02:04:43.521891 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.521839 2574 scope.go:117] "RemoveContainer" containerID="a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463" Jul 06 02:04:43.530504 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.530487 2574 scope.go:117] "RemoveContainer" containerID="9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb" Jul 06 02:04:43.531277 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:04:43.530822 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb\": container with ID starting with 9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb not found: ID does not exist" containerID="9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb" Jul 06 02:04:43.531277 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.530890 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb"} err="failed to get container status \"9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb\": rpc error: code = NotFound desc = could not find container \"9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb\": container with ID starting with 9c854f63cbeee57bc1d44351abf0ef46c00138e8117d49ed9024df8b6cccd1bb not found: ID does not exist" Jul 06 02:04:43.531277 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.530917 2574 scope.go:117] "RemoveContainer" containerID="a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463" Jul 06 02:04:43.531277 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:04:43.531212 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463\": container with ID starting with a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463 not found: ID does not exist" containerID="a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463" Jul 06 02:04:43.531277 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.531249 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463"} err="failed to get container status \"a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463\": rpc error: code = NotFound desc = could not find container \"a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463\": container with ID starting with a0a0b340ed4614068b0bcecda6ee6a0aa5aeff588fcf0463ff66a430f1eff463 not found: ID does not exist" Jul 06 02:04:43.533952 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.533931 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr"] Jul 06 02:04:43.538292 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:43.538270 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-782aa6-openshift-default-68b74dc5f7-m6ccr"] Jul 06 02:04:44.383168 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:44.383133 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" path="/var/lib/kubelet/pods/bb95a891-db20-40ac-a3fb-51e8dc3f97f5/volumes" Jul 06 02:04:46.815012 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.814976 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w"] Jul 06 02:04:46.815719 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.815695 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" Jul 06 02:04:46.815719 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.815721 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" Jul 06 02:04:46.815920 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.815733 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" Jul 06 02:04:46.815920 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.815741 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" Jul 06 02:04:46.815920 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.815850 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" Jul 06 02:04:46.815920 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.815887 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="11e9bfd3-f703-4463-ba2e-b5ed4526a990" containerName="istio-proxy" Jul 06 02:04:46.815920 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.815898 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb95a891-db20-40ac-a3fb-51e8dc3f97f5" containerName="istio-proxy" Jul 06 02:04:46.818024 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.818002 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.820313 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.820290 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d40f48-openshift-default-dockercfg-c8s74\"" Jul 06 02:04:46.830712 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.830689 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w"] Jul 06 02:04:46.959396 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959351 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-envoy\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.959396 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959396 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-data\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.959629 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959427 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpcwk\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-kube-api-access-rpcwk\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.959629 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959473 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-token\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.959629 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959511 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-credential-socket\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.959629 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959540 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-certs\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.959629 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959583 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-podinfo\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.959629 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959603 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-socket\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:46.959828 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:46.959643 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e52f005-3b02-4245-8cea-fb07acac97c2-istiod-ca-cert\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.060790 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.060746 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e52f005-3b02-4245-8cea-fb07acac97c2-istiod-ca-cert\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.060970 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.060922 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-envoy\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.060970 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.060961 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-data\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061097 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.060988 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpcwk\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-kube-api-access-rpcwk\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061097 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061039 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-token\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061097 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061088 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-credential-socket\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061251 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061120 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-certs\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061251 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061185 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-podinfo\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061251 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061219 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-socket\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061410 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061349 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-data\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061551 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061526 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e52f005-3b02-4245-8cea-fb07acac97c2-istiod-ca-cert\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061634 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061562 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-certs\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061634 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061586 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-socket\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.061703 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.061650 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-credential-socket\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.063601 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.063571 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-envoy\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.063993 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.063976 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-podinfo\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.070680 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.070655 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-token\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.070797 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.070744 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpcwk\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-kube-api-access-rpcwk\") pod \"e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.130003 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.129965 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:47.277123 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.277088 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w"] Jul 06 02:04:47.278902 ip-10-0-133-193 kubenswrapper[2574]: W0706 02:04:47.278838 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e52f005_3b02_4245_8cea_fb07acac97c2.slice/crio-deeda86fd4356d97394b2cc88e00746a8d7fdaa80fcad03c4c42359960533425 WatchSource:0}: Error finding container deeda86fd4356d97394b2cc88e00746a8d7fdaa80fcad03c4c42359960533425: Status 404 returned error can't find the container with id deeda86fd4356d97394b2cc88e00746a8d7fdaa80fcad03c4c42359960533425 Jul 06 02:04:47.281340 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.281303 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:47.281421 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.281386 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:47.281457 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.281425 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:47.540469 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.540425 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" event={"ID":"0e52f005-3b02-4245-8cea-fb07acac97c2","Type":"ContainerStarted","Data":"3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a"} Jul 06 02:04:47.540632 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.540497 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" event={"ID":"0e52f005-3b02-4245-8cea-fb07acac97c2","Type":"ContainerStarted","Data":"deeda86fd4356d97394b2cc88e00746a8d7fdaa80fcad03c4c42359960533425"} Jul 06 02:04:47.563048 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:47.562990 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" podStartSLOduration=1.562968567 podStartE2EDuration="1.562968567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:04:47.559088656 +0000 UTC m=+2285.694379160" watchObservedRunningTime="2026-07-06 02:04:47.562968567 +0000 UTC m=+2285.698259072" Jul 06 02:04:48.130948 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:48.130910 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:48.136530 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:48.136505 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:48.548418 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:48.548326 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:48.554108 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:48.554035 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:04:57.541074 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:57.541021 2574 patch_prober.go:28] interesting pod/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 02:04:57.542575 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:57.541106 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 02:04:57.616317 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:57.616282 2574 generic.go:356] "Generic (PLEG): container finished" podID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerID="dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68" exitCode=0 Jul 06 02:04:57.616483 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:57.616350 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" event={"ID":"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb","Type":"ContainerDied","Data":"dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68"} Jul 06 02:04:57.616815 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:57.616797 2574 scope.go:117] "RemoveContainer" containerID="dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68" Jul 06 02:04:57.617947 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:57.617917 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:57.618067 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:57.618006 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:57.618067 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:57.618055 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:04:58.624346 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:58.624315 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" event={"ID":"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb","Type":"ContainerStarted","Data":"e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad"} Jul 06 02:04:58.902717 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:58.902611 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:04:58.902923 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:58.902755 2574 patch_prober.go:28] interesting pod/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 06 02:04:58.902923 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:58.902799 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 06 02:04:59.903326 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:59.903286 2574 patch_prober.go:28] interesting pod/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 06 02:04:59.903985 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:04:59.903952 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 06 02:05:00.903518 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:00.903474 2574 patch_prober.go:28] interesting pod/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 06 02:05:00.904203 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:00.904164 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 06 02:05:01.903262 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:01.903219 2574 patch_prober.go:28] interesting pod/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 06 02:05:01.903687 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:01.903652 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 06 02:05:02.906265 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:02.906231 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:05:02.906780 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:02.906651 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:05:02.907406 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:02.907385 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:05:04.589051 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:04.589010 2574 patch_prober.go:28] interesting pod/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 02:05:04.589459 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:04.589083 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 02:05:05.672830 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:05.672735 2574 generic.go:356] "Generic (PLEG): container finished" podID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerID="3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a" exitCode=0 Jul 06 02:05:05.672830 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:05.672808 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" event={"ID":"0e52f005-3b02-4245-8cea-fb07acac97c2","Type":"ContainerDied","Data":"3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a"} Jul 06 02:05:05.673381 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:05.673280 2574 scope.go:117] "RemoveContainer" containerID="3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a" Jul 06 02:05:05.674420 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:05.674383 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:05.674506 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:05.674465 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:05.674506 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:05.674496 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:06.685584 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:06.685542 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" event={"ID":"0e52f005-3b02-4245-8cea-fb07acac97c2","Type":"ContainerStarted","Data":"97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd"} Jul 06 02:05:07.130635 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:07.130594 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:05:07.130827 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:07.130808 2574 patch_prober.go:28] interesting pod/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 06 02:05:07.130919 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:07.130897 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 06 02:05:08.131416 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:08.131373 2574 patch_prober.go:28] interesting pod/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 06 02:05:08.131818 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:08.131451 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 06 02:05:08.822585 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:08.822554 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs"] Jul 06 02:05:08.822814 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:08.822780 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" containerID="cri-o://e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad" gracePeriod=30 Jul 06 02:05:09.130791 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:09.130692 2574 patch_prober.go:28] interesting pod/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 06 02:05:09.130791 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:09.130767 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 06 02:05:10.131074 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:10.131044 2574 patch_prober.go:28] interesting pod/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 06 02:05:10.131455 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:10.131098 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 06 02:05:11.133823 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:11.133791 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:05:11.134265 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:11.134184 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:05:11.134966 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:11.134945 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:05:14.076732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.076707 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:05:14.154853 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.154751 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-credential-socket\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.154853 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.154817 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-data\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.155131 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.154910 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istiod-ca-cert\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.155131 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.154949 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-certs\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.155131 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.154986 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-token\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.155131 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155029 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-socket\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.155131 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155092 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-envoy\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.155478 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155139 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q6h7n\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-kube-api-access-q6h7n\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.155478 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155174 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-podinfo\") pod \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\" (UID: \"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb\") " Jul 06 02:05:14.155478 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155163 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:14.155478 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155304 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:14.155478 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155382 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:05:14.155731 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155588 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.155731 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155606 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.155731 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155617 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.155731 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155723 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:14.156129 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.155785 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:14.157381 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.157353 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:14.157666 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.157453 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 02:05:14.158000 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.157971 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-kube-api-access-q6h7n" (OuterVolumeSpecName: "kube-api-access-q6h7n") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "kube-api-access-q6h7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:05:14.158000 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.157978 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" (UID: "dc9db81a-cc6b-4aa1-928f-4f25027ac0bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:05:14.256197 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.256166 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.256197 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.256193 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.256197 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.256203 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.256425 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.256213 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.256425 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.256222 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6h7n\" (UniqueName: \"kubernetes.io/projected/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-kube-api-access-q6h7n\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.256425 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.256232 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:14.739066 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.739032 2574 generic.go:356] "Generic (PLEG): container finished" podID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerID="e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad" exitCode=0 Jul 06 02:05:14.739256 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.739080 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" Jul 06 02:05:14.739256 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.739109 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" event={"ID":"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb","Type":"ContainerDied","Data":"e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad"} Jul 06 02:05:14.739256 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.739153 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs" event={"ID":"dc9db81a-cc6b-4aa1-928f-4f25027ac0bb","Type":"ContainerDied","Data":"06dc27d5cd3ac77b269c402506881137fcf388c7de3f4e2d667145035d6d2617"} Jul 06 02:05:14.739256 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.739174 2574 scope.go:117] "RemoveContainer" containerID="e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad" Jul 06 02:05:14.749299 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.749276 2574 scope.go:117] "RemoveContainer" containerID="dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68" Jul 06 02:05:14.758357 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.758324 2574 scope.go:117] "RemoveContainer" containerID="e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad" Jul 06 02:05:14.758617 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:05:14.758599 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad\": container with ID starting with e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad not found: ID does not exist" containerID="e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad" Jul 06 02:05:14.758686 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.758627 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad"} err="failed to get container status \"e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad\": rpc error: code = NotFound desc = could not find container \"e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad\": container with ID starting with e93ba714499d3cb43c94917954bc796abe7553d580e7e98b0ca99e4c7441a2ad not found: ID does not exist" Jul 06 02:05:14.758686 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.758646 2574 scope.go:117] "RemoveContainer" containerID="dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68" Jul 06 02:05:14.758994 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:05:14.758973 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68\": container with ID starting with dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68 not found: ID does not exist" containerID="dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68" Jul 06 02:05:14.759060 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.759000 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68"} err="failed to get container status \"dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68\": rpc error: code = NotFound desc = could not find container \"dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68\": container with ID starting with dc0bd957b67eb59c7c0411a2a1e772b62fb3c66660b6c2bc3b54da374174bf68 not found: ID does not exist" Jul 06 02:05:14.764164 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.764140 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs"] Jul 06 02:05:14.771554 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:14.771532 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-0cd54e-openshift-default-6d8d75f4c7-f7hcs"] Jul 06 02:05:16.067256 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.067222 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w"] Jul 06 02:05:16.067679 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.067534 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" containerID="cri-o://97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd" gracePeriod=30 Jul 06 02:05:16.382618 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.382585 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" path="/var/lib/kubelet/pods/dc9db81a-cc6b-4aa1-928f-4f25027ac0bb/volumes" Jul 06 02:05:16.942166 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.942135 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd"] Jul 06 02:05:16.942627 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.942613 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" Jul 06 02:05:16.942675 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.942629 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" Jul 06 02:05:16.942675 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.942639 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" Jul 06 02:05:16.942675 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.942645 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" Jul 06 02:05:16.942770 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.942713 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" Jul 06 02:05:16.944996 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.944980 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:16.947312 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.947288 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-388824-openshift-default-dockercfg-lxp9f\"" Jul 06 02:05:16.959233 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:16.959209 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd"] Jul 06 02:05:17.083657 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083624 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-envoy\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.084069 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083671 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-data\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.084069 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083708 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042bc62b-0da9-4986-8903-462a941a477f-istiod-ca-cert\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.084069 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083740 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-istio-token\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.084069 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083765 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-socket\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.084069 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083806 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-certs\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.084069 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083822 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-credential-socket\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.084069 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083850 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042bc62b-0da9-4986-8903-462a941a477f-istio-podinfo\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.084069 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.083896 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vrc4\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-kube-api-access-7vrc4\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185052 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185015 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-data\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185239 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042bc62b-0da9-4986-8903-462a941a477f-istiod-ca-cert\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185239 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-istio-token\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185239 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185139 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-socket\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185239 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185209 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-certs\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185432 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185241 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-credential-socket\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185432 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185281 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042bc62b-0da9-4986-8903-462a941a477f-istio-podinfo\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185432 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185306 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vrc4\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-kube-api-access-7vrc4\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185594 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185458 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-envoy\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185594 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185500 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-data\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185734 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185587 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-certs\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185734 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185649 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-credential-socket\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.185847 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.185825 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-socket\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.186056 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.186027 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042bc62b-0da9-4986-8903-462a941a477f-istiod-ca-cert\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.187791 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.187765 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-envoy\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.188024 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.188006 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042bc62b-0da9-4986-8903-462a941a477f-istio-podinfo\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.193183 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.193123 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-istio-token\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.193266 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.193215 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vrc4\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-kube-api-access-7vrc4\") pod \"e2e-shared-a-388824-openshift-default-54bff685f-nl6kd\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.255545 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.255513 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:17.401927 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.401894 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd"] Jul 06 02:05:17.403044 ip-10-0-133-193 kubenswrapper[2574]: W0706 02:05:17.403010 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042bc62b_0da9_4986_8903_462a941a477f.slice/crio-0cdb7c7fcaa617f7f7f2ac31308af8780b38a29e6c8a227721410da63bc6d062 WatchSource:0}: Error finding container 0cdb7c7fcaa617f7f7f2ac31308af8780b38a29e6c8a227721410da63bc6d062: Status 404 returned error can't find the container with id 0cdb7c7fcaa617f7f7f2ac31308af8780b38a29e6c8a227721410da63bc6d062 Jul 06 02:05:17.405665 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.405631 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:17.405761 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.405711 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:17.405761 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.405753 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:17.760275 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.760188 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" event={"ID":"042bc62b-0da9-4986-8903-462a941a477f","Type":"ContainerStarted","Data":"d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6"} Jul 06 02:05:17.760275 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.760245 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" event={"ID":"042bc62b-0da9-4986-8903-462a941a477f","Type":"ContainerStarted","Data":"0cdb7c7fcaa617f7f7f2ac31308af8780b38a29e6c8a227721410da63bc6d062"} Jul 06 02:05:17.780047 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:17.779980 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" podStartSLOduration=1.779964796 podStartE2EDuration="1.779964796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:05:17.77883571 +0000 UTC m=+2315.914126216" watchObservedRunningTime="2026-07-06 02:05:17.779964796 +0000 UTC m=+2315.915255300" Jul 06 02:05:18.255939 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:18.255901 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:18.261520 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:18.261494 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:18.767325 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:18.767290 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:18.768240 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:18.768222 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:21.334975 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.334915 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:05:21.428757 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.428717 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-socket\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.428991 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.428805 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-envoy\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.428991 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.428913 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-data\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.428991 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.428939 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-token\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.428991 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.428950 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:21.428991 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.428990 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-credential-socket\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.429260 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.429021 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpcwk\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-kube-api-access-rpcwk\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.429260 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.429050 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-podinfo\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.429260 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.429088 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-certs\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.429260 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.429133 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e52f005-3b02-4245-8cea-fb07acac97c2-istiod-ca-cert\") pod \"0e52f005-3b02-4245-8cea-fb07acac97c2\" (UID: \"0e52f005-3b02-4245-8cea-fb07acac97c2\") " Jul 06 02:05:21.429468 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.429304 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:21.429468 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.429346 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:21.429602 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.429583 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e52f005-3b02-4245-8cea-fb07acac97c2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:05:21.430068 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.430046 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.430179 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.430075 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.430179 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.430092 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.430179 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.430152 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:21.431488 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.431460 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 02:05:21.431624 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.431602 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-kube-api-access-rpcwk" (OuterVolumeSpecName: "kube-api-access-rpcwk") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "kube-api-access-rpcwk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:05:21.431714 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.431695 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:05:21.431763 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.431723 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e52f005-3b02-4245-8cea-fb07acac97c2" (UID: "0e52f005-3b02-4245-8cea-fb07acac97c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:05:21.530984 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.530947 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.530984 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.530978 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.530984 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.530991 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.531229 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.531003 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpcwk\" (UniqueName: \"kubernetes.io/projected/0e52f005-3b02-4245-8cea-fb07acac97c2-kube-api-access-rpcwk\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.531229 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.531015 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e52f005-3b02-4245-8cea-fb07acac97c2-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.531229 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.531027 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e52f005-3b02-4245-8cea-fb07acac97c2-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:05:21.788943 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.788825 2574 generic.go:356] "Generic (PLEG): container finished" podID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerID="97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd" exitCode=0 Jul 06 02:05:21.788943 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.788917 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" Jul 06 02:05:21.788943 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.788914 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" event={"ID":"0e52f005-3b02-4245-8cea-fb07acac97c2","Type":"ContainerDied","Data":"97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd"} Jul 06 02:05:21.789209 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.788984 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w" event={"ID":"0e52f005-3b02-4245-8cea-fb07acac97c2","Type":"ContainerDied","Data":"deeda86fd4356d97394b2cc88e00746a8d7fdaa80fcad03c4c42359960533425"} Jul 06 02:05:21.789209 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.789008 2574 scope.go:117] "RemoveContainer" containerID="97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd" Jul 06 02:05:21.804266 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.804241 2574 scope.go:117] "RemoveContainer" containerID="3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a" Jul 06 02:05:21.814849 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.814825 2574 scope.go:117] "RemoveContainer" containerID="97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd" Jul 06 02:05:21.815227 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:05:21.815205 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd\": container with ID starting with 97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd not found: ID does not exist" containerID="97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd" Jul 06 02:05:21.815313 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.815236 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd"} err="failed to get container status \"97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd\": rpc error: code = NotFound desc = could not find container \"97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd\": container with ID starting with 97d7862c290172d1235b423eed25ad7b3800d2e88dbc3dccdddbf4ed8c1ecbbd not found: ID does not exist" Jul 06 02:05:21.815313 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.815257 2574 scope.go:117] "RemoveContainer" containerID="3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a" Jul 06 02:05:21.815551 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:05:21.815536 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a\": container with ID starting with 3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a not found: ID does not exist" containerID="3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a" Jul 06 02:05:21.815590 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.815557 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a"} err="failed to get container status \"3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a\": rpc error: code = NotFound desc = could not find container \"3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a\": container with ID starting with 3372519779ecac784284c80aa6b06df35d9df518e14dcb27580b9053dd20133a not found: ID does not exist" Jul 06 02:05:21.816068 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.816049 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w"] Jul 06 02:05:21.822410 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:21.822381 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d40f48-openshift-default-9756bc5fc-cl77w"] Jul 06 02:05:22.383568 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:22.383536 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" path="/var/lib/kubelet/pods/0e52f005-3b02-4245-8cea-fb07acac97c2/volumes" Jul 06 02:05:24.921043 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:24.921003 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn"] Jul 06 02:05:24.921498 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:24.921484 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" Jul 06 02:05:24.921545 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:24.921501 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" Jul 06 02:05:24.921596 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:24.921586 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" Jul 06 02:05:24.921632 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:24.921600 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc9db81a-cc6b-4aa1-928f-4f25027ac0bb" containerName="istio-proxy" Jul 06 02:05:24.924590 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:24.924570 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:24.927220 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:24.927195 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3d280d-openshift-default-dockercfg-n42wv\"" Jul 06 02:05:24.936087 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:24.936056 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn"] Jul 06 02:05:25.069633 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.069596 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-credential-socket\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.069633 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.069636 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-data\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.069977 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.069657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istiod-ca-cert\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.069977 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.069727 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-token\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.069977 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.069800 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-podinfo\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.069977 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.069826 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-envoy\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.069977 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.069931 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-socket\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.070194 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.070015 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-certs\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.070194 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.070061 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svlxz\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-kube-api-access-svlxz\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.171685 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171595 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-credential-socket\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.171685 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171637 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-data\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.171685 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171661 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istiod-ca-cert\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172025 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171695 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-token\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172025 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171750 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-podinfo\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172025 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171781 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-envoy\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172025 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171832 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-socket\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172025 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171935 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-certs\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172025 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.171981 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svlxz\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-kube-api-access-svlxz\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172344 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.172261 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-credential-socket\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172344 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.172291 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-socket\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172445 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.172351 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-certs\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.172445 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.172422 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-data\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.173120 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.173098 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istiod-ca-cert\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.174346 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.174323 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-envoy\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.174517 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.174502 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-podinfo\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.184664 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.184636 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-token\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.184832 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.184815 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svlxz\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-kube-api-access-svlxz\") pod \"e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.236818 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.236776 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:25.391409 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.391381 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn"] Jul 06 02:05:25.393047 ip-10-0-133-193 kubenswrapper[2574]: W0706 02:05:25.393018 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca8e0225_d834_4ddc_bc29_ac680d9d3bc4.slice/crio-0eea994b6807c03fa6cc0bc78ba5a149f33b6289b04db077ea073c2b2560bc85 WatchSource:0}: Error finding container 0eea994b6807c03fa6cc0bc78ba5a149f33b6289b04db077ea073c2b2560bc85: Status 404 returned error can't find the container with id 0eea994b6807c03fa6cc0bc78ba5a149f33b6289b04db077ea073c2b2560bc85 Jul 06 02:05:25.395280 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.395250 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:25.395360 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.395313 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:25.395401 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.395344 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:25.820452 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.820415 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" event={"ID":"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4","Type":"ContainerStarted","Data":"7e45e4cdd9cfc1c3defe03bff78172ae017dd882fb080a7498fa90a9865a15cb"} Jul 06 02:05:25.820649 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.820471 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" event={"ID":"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4","Type":"ContainerStarted","Data":"0eea994b6807c03fa6cc0bc78ba5a149f33b6289b04db077ea073c2b2560bc85"} Jul 06 02:05:25.843336 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:25.843278 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" podStartSLOduration=1.843256201 podStartE2EDuration="1.843256201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 02:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:05:25.840078385 +0000 UTC m=+2323.975368890" watchObservedRunningTime="2026-07-06 02:05:25.843256201 +0000 UTC m=+2323.978546706" Jul 06 02:05:26.237622 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:26.237539 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:26.242803 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:26.242774 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:26.826576 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:26.826547 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:26.827517 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:26.827494 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:33.769124 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:33.769022 2574 patch_prober.go:28] interesting pod/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 06 02:05:33.769124 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:33.769082 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 06 02:05:33.873644 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:33.873608 2574 generic.go:356] "Generic (PLEG): container finished" podID="042bc62b-0da9-4986-8903-462a941a477f" containerID="d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6" exitCode=0 Jul 06 02:05:33.873822 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:33.873684 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" event={"ID":"042bc62b-0da9-4986-8903-462a941a477f","Type":"ContainerDied","Data":"d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6"} Jul 06 02:05:33.874192 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:33.874174 2574 scope.go:117] "RemoveContainer" containerID="d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6" Jul 06 02:05:33.875334 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:33.875304 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:33.875410 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:33.875377 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:33.875450 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:33.875410 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:34.897959 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:34.897913 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" event={"ID":"042bc62b-0da9-4986-8903-462a941a477f","Type":"ContainerStarted","Data":"ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf"} Jul 06 02:05:35.256062 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:35.255851 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:35.256307 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:35.256286 2574 patch_prober.go:28] interesting pod/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 06 02:05:35.256372 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:35.256337 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 06 02:05:36.256319 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:36.256286 2574 patch_prober.go:28] interesting pod/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 06 02:05:36.256729 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:36.256341 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 06 02:05:37.256673 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:37.256634 2574 patch_prober.go:28] interesting pod/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 06 02:05:37.257093 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:37.256709 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 06 02:05:38.256133 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:38.256096 2574 patch_prober.go:28] interesting pod/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 06 02:05:38.256354 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:38.256160 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 06 02:05:39.260319 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:39.260289 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:39.260783 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:39.260773 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:39.261503 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:39.261480 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:05:41.895344 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:41.894647 2574 patch_prober.go:28] interesting pod/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": read tcp 10.133.0.2:40840->10.133.0.64:15021: read: connection reset by peer" start-of-body= Jul 06 02:05:41.895344 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:41.894713 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": read tcp 10.133.0.2:40840->10.133.0.64:15021: read: connection reset by peer" Jul 06 02:05:41.949052 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:41.949020 2574 generic.go:356] "Generic (PLEG): container finished" podID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerID="7e45e4cdd9cfc1c3defe03bff78172ae017dd882fb080a7498fa90a9865a15cb" exitCode=0 Jul 06 02:05:41.949203 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:41.949093 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" event={"ID":"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4","Type":"ContainerDied","Data":"7e45e4cdd9cfc1c3defe03bff78172ae017dd882fb080a7498fa90a9865a15cb"} Jul 06 02:05:41.949551 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:41.949533 2574 scope.go:117] "RemoveContainer" containerID="7e45e4cdd9cfc1c3defe03bff78172ae017dd882fb080a7498fa90a9865a15cb" Jul 06 02:05:41.950654 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:41.950624 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:41.950740 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:41.950725 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:41.950784 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:41.950773 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 02:05:42.957428 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:42.957339 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" event={"ID":"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4","Type":"ContainerStarted","Data":"be85d4b4e4ed73697e37d1d13db8d3cf8f4f3ff69b094d167d4b7c883a623848"} Jul 06 02:05:43.237436 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:43.237336 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:43.237689 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:43.237669 2574 patch_prober.go:28] interesting pod/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 06 02:05:43.237735 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:43.237715 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 06 02:05:44.238106 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:44.238069 2574 patch_prober.go:28] interesting pod/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 06 02:05:44.238533 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:44.238124 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 06 02:05:45.237509 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:45.237469 2574 patch_prober.go:28] interesting pod/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 06 02:05:45.237695 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:45.237533 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 06 02:05:46.238109 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:46.238075 2574 patch_prober.go:28] interesting pod/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 06 02:05:46.238525 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:46.238139 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 06 02:05:47.240875 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:47.240822 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:47.241281 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:47.241108 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:05:47.241749 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:05:47.241724 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:06:42.543182 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:06:42.543149 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 02:06:42.551909 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:06:42.551851 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 02:06:42.554498 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:06:42.554472 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 02:06:42.561966 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:06:42.561948 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 02:11:30.125440 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:30.125399 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd"] Jul 06 02:11:30.128025 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:30.125614 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" containerID="cri-o://ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf" gracePeriod=30 Jul 06 02:11:35.381553 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.381530 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:11:35.514808 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.514720 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-socket\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.514808 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.514756 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-envoy\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.514808 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.514783 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-credential-socket\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.515137 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.514830 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-certs\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.515137 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.514898 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vrc4\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-kube-api-access-7vrc4\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.515137 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.514971 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-data\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.515137 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515021 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-istio-token\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.515137 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515049 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042bc62b-0da9-4986-8903-462a941a477f-istiod-ca-cert\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.515137 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515077 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:35.515137 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515112 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042bc62b-0da9-4986-8903-462a941a477f-istio-podinfo\") pod \"042bc62b-0da9-4986-8903-462a941a477f\" (UID: \"042bc62b-0da9-4986-8903-462a941a477f\") " Jul 06 02:11:35.515510 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515138 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:35.515510 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515240 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:35.515510 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515480 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/042bc62b-0da9-4986-8903-462a941a477f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:11:35.515674 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515628 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042bc62b-0da9-4986-8903-462a941a477f-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:35.515674 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515646 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:35.515674 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515660 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:35.515828 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515673 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:35.515945 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.515916 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-data" (OuterVolumeSpecName: "istio-data") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:35.517638 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.517616 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:35.517760 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.517643 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-istio-token" (OuterVolumeSpecName: "istio-token") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:11:35.518178 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.518155 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-kube-api-access-7vrc4" (OuterVolumeSpecName: "kube-api-access-7vrc4") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "kube-api-access-7vrc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:11:35.518178 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.518164 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/042bc62b-0da9-4986-8903-462a941a477f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "042bc62b-0da9-4986-8903-462a941a477f" (UID: "042bc62b-0da9-4986-8903-462a941a477f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 02:11:35.616933 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.616904 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vrc4\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-kube-api-access-7vrc4\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:35.616933 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.616930 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:35.616933 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.616940 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042bc62b-0da9-4986-8903-462a941a477f-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:35.617150 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.616949 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042bc62b-0da9-4986-8903-462a941a477f-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:35.617150 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:35.616958 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042bc62b-0da9-4986-8903-462a941a477f-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:36.049808 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.049767 2574 generic.go:356] "Generic (PLEG): container finished" podID="042bc62b-0da9-4986-8903-462a941a477f" containerID="ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf" exitCode=0 Jul 06 02:11:36.049992 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.049816 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" event={"ID":"042bc62b-0da9-4986-8903-462a941a477f","Type":"ContainerDied","Data":"ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf"} Jul 06 02:11:36.049992 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.049820 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" Jul 06 02:11:36.049992 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.049885 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd" event={"ID":"042bc62b-0da9-4986-8903-462a941a477f","Type":"ContainerDied","Data":"0cdb7c7fcaa617f7f7f2ac31308af8780b38a29e6c8a227721410da63bc6d062"} Jul 06 02:11:36.049992 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.049908 2574 scope.go:117] "RemoveContainer" containerID="ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf" Jul 06 02:11:36.059960 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.059943 2574 scope.go:117] "RemoveContainer" containerID="d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6" Jul 06 02:11:36.069119 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.069100 2574 scope.go:117] "RemoveContainer" containerID="ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf" Jul 06 02:11:36.069406 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:11:36.069386 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf\": container with ID starting with ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf not found: ID does not exist" containerID="ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf" Jul 06 02:11:36.069480 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.069416 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf"} err="failed to get container status \"ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf\": rpc error: code = NotFound desc = could not find container \"ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf\": container with ID starting with ab3754703075e80c104387b9a36909135dc9b6b1c57ea757fce6c25d3a982caf not found: ID does not exist" Jul 06 02:11:36.069480 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.069433 2574 scope.go:117] "RemoveContainer" containerID="d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6" Jul 06 02:11:36.069690 ip-10-0-133-193 kubenswrapper[2574]: E0706 02:11:36.069669 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6\": container with ID starting with d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6 not found: ID does not exist" containerID="d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6" Jul 06 02:11:36.069745 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.069700 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6"} err="failed to get container status \"d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6\": rpc error: code = NotFound desc = could not find container \"d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6\": container with ID starting with d6e674238b8da8d7072b611319bbb60e40dc5b67651325c1a30b3fc9887858e6 not found: ID does not exist" Jul 06 02:11:36.071668 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.071649 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd"] Jul 06 02:11:36.074538 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.074519 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-388824-openshift-default-54bff685f-nl6kd"] Jul 06 02:11:36.382999 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:36.382970 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="042bc62b-0da9-4986-8903-462a941a477f" path="/var/lib/kubelet/pods/042bc62b-0da9-4986-8903-462a941a477f/volumes" Jul 06 02:11:37.142254 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:37.142222 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn"] Jul 06 02:11:37.142504 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:37.142481 2574 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" containerID="cri-o://be85d4b4e4ed73697e37d1d13db8d3cf8f4f3ff69b094d167d4b7c883a623848" gracePeriod=30 Jul 06 02:11:42.406348 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.406286 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:11:42.485033 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.484998 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svlxz\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-kube-api-access-svlxz\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485033 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485039 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-certs\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485281 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485076 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istiod-ca-cert\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485281 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485119 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-token\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485281 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485133 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-envoy\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485281 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485165 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-data\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485281 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485231 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-credential-socket\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485281 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485274 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-socket\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485605 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485303 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-podinfo\") pod \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\" (UID: \"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4\") " Jul 06 02:11:42.485605 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485416 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:42.485605 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485510 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 02:11:42.485771 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485606 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:42.485771 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485733 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:42.485912 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485832 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-credential-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.485912 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485850 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-socket\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.486195 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.485931 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-workload-certs\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.486195 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.486041 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istiod-ca-cert\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.486310 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.486239 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:42.487554 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.487524 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 02:11:42.487668 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.487576 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 02:11:42.487838 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.487814 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-kube-api-access-svlxz" (OuterVolumeSpecName: "kube-api-access-svlxz") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "kube-api-access-svlxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:11:42.487838 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.487822 2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" (UID: "ca8e0225-d834-4ddc-bc29-ac680d9d3bc4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 02:11:42.558540 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.558516 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f669l_3e7ae9b0-50f2-48f3-b07c-d5d58badcee7/manager/0.log" Jul 06 02:11:42.586664 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.586631 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svlxz\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-kube-api-access-svlxz\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.586664 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.586668 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-token\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.586912 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.586682 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-envoy\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.586912 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.586690 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-data\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.586912 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.586697 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4-istio-podinfo\") on node \"ip-10-0-133-193.ec2.internal\" DevicePath \"\"" Jul 06 02:11:42.593608 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.593585 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 02:11:42.600746 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.600729 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 02:11:42.605016 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.604999 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 02:11:42.611851 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.611834 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pbj9_682756aa-6265-455c-8c8e-81e95817fd8a/ovn-acl-logging/0.log" Jul 06 02:11:42.683560 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.683491 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6ddd7f5dc5-b24gf_ea7c9960-615e-4a5f-9011-3377d60cdd14/maas-api/0.log" Jul 06 02:11:42.725898 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.725850 2574 scope.go:117] "RemoveContainer" containerID="7e45e4cdd9cfc1c3defe03bff78172ae017dd882fb080a7498fa90a9865a15cb" Jul 06 02:11:42.735408 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.735391 2574 scope.go:117] "RemoveContainer" containerID="be85d4b4e4ed73697e37d1d13db8d3cf8f4f3ff69b094d167d4b7c883a623848" Jul 06 02:11:42.916933 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:42.916905 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29721720-5v9h2_be3e159d-d3c4-4954-bb8b-0ee8fa05719e/cleanup/0.log" Jul 06 02:11:43.042918 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.042833 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d6f449ff5-jszv6_951492ed-9116-467d-ab29-d0cb52e555dd/manager/0.log" Jul 06 02:11:43.096783 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.096750 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" event={"ID":"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4","Type":"ContainerDied","Data":"be85d4b4e4ed73697e37d1d13db8d3cf8f4f3ff69b094d167d4b7c883a623848"} Jul 06 02:11:43.096783 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.096784 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" Jul 06 02:11:43.097003 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.096839 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn" event={"ID":"ca8e0225-d834-4ddc-bc29-ac680d9d3bc4","Type":"ContainerDied","Data":"0eea994b6807c03fa6cc0bc78ba5a149f33b6289b04db077ea073c2b2560bc85"} Jul 06 02:11:43.128347 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.127982 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn"] Jul 06 02:11:43.131477 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.131455 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3d280d-openshift-default-f46f78747-j2bdn"] Jul 06 02:11:43.545989 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.545966 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656987b587-lx7xt_5439a382-34f1-4d8f-bdf3-48db422e2539/copy-manifests/0.log" Jul 06 02:11:43.552098 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.552072 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656987b587-lx7xt_5439a382-34f1-4d8f-bdf3-48db422e2539/manager/0.log" Jul 06 02:11:43.785566 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:43.785541 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-8m2q9_433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9/postgres/0.log" Jul 06 02:11:44.383041 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.383010 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" path="/var/lib/kubelet/pods/ca8e0225-d834-4ddc-bc29-ac680d9d3bc4/volumes" Jul 06 02:11:44.593251 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.593223 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg_edf02925-4a13-423d-bc70-f477402fd30e/util/0.log" Jul 06 02:11:44.600345 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.600322 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg_edf02925-4a13-423d-bc70-f477402fd30e/pull/0.log" Jul 06 02:11:44.606223 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.606207 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg_edf02925-4a13-423d-bc70-f477402fd30e/extract/0.log" Jul 06 02:11:44.717189 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.717124 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh_81ed29d4-4135-429a-98cc-c562c1d3fabe/extract/0.log" Jul 06 02:11:44.722483 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.722468 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh_81ed29d4-4135-429a-98cc-c562c1d3fabe/util/0.log" Jul 06 02:11:44.727914 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.727896 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh_81ed29d4-4135-429a-98cc-c562c1d3fabe/pull/0.log" Jul 06 02:11:44.836202 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.836181 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm_5279d4cb-7a8c-4b59-a45a-ac16635c2216/pull/0.log" Jul 06 02:11:44.842123 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.842106 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm_5279d4cb-7a8c-4b59-a45a-ac16635c2216/extract/0.log" Jul 06 02:11:44.847912 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.847890 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm_5279d4cb-7a8c-4b59-a45a-ac16635c2216/util/0.log" Jul 06 02:11:44.960265 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.960242 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw_35365362-fd3d-4dcd-a422-1aa8acaa9dfb/util/0.log" Jul 06 02:11:44.966565 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.966542 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw_35365362-fd3d-4dcd-a422-1aa8acaa9dfb/pull/0.log" Jul 06 02:11:44.972768 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:44.972709 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw_35365362-fd3d-4dcd-a422-1aa8acaa9dfb/extract/0.log" Jul 06 02:11:45.209709 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:45.209677 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dkvhg_56da7f46-ffea-4848-8458-dec5bf391c6f/manager/0.log" Jul 06 02:11:45.317203 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:45.317128 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4nbj5_e12b1f19-8e4c-4507-a8f7-d485b96ebb63/manager/0.log" Jul 06 02:11:46.242410 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:46.242384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46nlc7_b11c570f-2368-42f1-a601-af6747c34507/istio-proxy/0.log" Jul 06 02:11:46.365437 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:46.365410 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-stbx7_478f87cf-c161-4a86-b3dd-2d71dbf45de3/discovery/0.log" Jul 06 02:11:46.606408 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:46.606384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f644fb7f6-ldnm4_e570dbbc-7813-4a2b-b689-a208edf74ebc/kube-auth-proxy/0.log" Jul 06 02:11:46.725117 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:46.725093 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8wf7_abde296b-bf0d-45e4-8db4-d79ee2e60e0f/istio-proxy/1.log" Jul 06 02:11:46.846625 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:46.846600 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z9qkm_b07852b5-6abf-422e-a7d0-699270afbb06/payload-pre-processing/0.log" Jul 06 02:11:47.088053 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:47.088020 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7995fbff9c-klmcz_5fe2ae0a-62b0-4938-a271-bb810c1d533c/router/0.log" Jul 06 02:11:47.448400 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:47.448321 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-s7kqb_ad343c45-028e-45eb-a728-f379f458f310/main/0.log" Jul 06 02:11:47.931381 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:47.931356 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-grxs6_7b12ac32-0034-452d-9656-927d3ba0a65a/main/0.log" Jul 06 02:11:54.746680 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:54.746646 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nzcfc_ad5aa4e4-5a9f-4f11-a29d-90c6755b2b4e/global-pull-secret-syncer/0.log" Jul 06 02:11:54.922097 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:54.922069 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qws5x_da1603f3-dd4b-4993-9730-75f00719033b/konnectivity-agent/0.log" Jul 06 02:11:54.994839 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:54.994812 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-193.ec2.internal_4ae2047bf8edfd2708b684a75d4f25bd/haproxy/0.log" Jul 06 02:11:59.625767 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.625732 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg_edf02925-4a13-423d-bc70-f477402fd30e/extract/0.log" Jul 06 02:11:59.646868 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.646842 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg_edf02925-4a13-423d-bc70-f477402fd30e/util/0.log" Jul 06 02:11:59.667705 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.667679 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xhfhg_edf02925-4a13-423d-bc70-f477402fd30e/pull/0.log" Jul 06 02:11:59.694142 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.694116 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh_81ed29d4-4135-429a-98cc-c562c1d3fabe/extract/0.log" Jul 06 02:11:59.720211 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.720181 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh_81ed29d4-4135-429a-98cc-c562c1d3fabe/util/0.log" Jul 06 02:11:59.745620 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.745549 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0785vh_81ed29d4-4135-429a-98cc-c562c1d3fabe/pull/0.log" Jul 06 02:11:59.772674 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.772648 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm_5279d4cb-7a8c-4b59-a45a-ac16635c2216/extract/0.log" Jul 06 02:11:59.795170 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.795147 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm_5279d4cb-7a8c-4b59-a45a-ac16635c2216/util/0.log" Jul 06 02:11:59.815960 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.815939 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735gddm_5279d4cb-7a8c-4b59-a45a-ac16635c2216/pull/0.log" Jul 06 02:11:59.842113 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.842091 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw_35365362-fd3d-4dcd-a422-1aa8acaa9dfb/extract/0.log" Jul 06 02:11:59.864879 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.864837 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw_35365362-fd3d-4dcd-a422-1aa8acaa9dfb/util/0.log" Jul 06 02:11:59.882505 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:11:59.882484 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef186kpw_35365362-fd3d-4dcd-a422-1aa8acaa9dfb/pull/0.log" Jul 06 02:12:00.408617 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:00.404821 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-dkvhg_56da7f46-ffea-4848-8458-dec5bf391c6f/manager/0.log" Jul 06 02:12:00.430985 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:00.430956 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4nbj5_e12b1f19-8e4c-4507-a8f7-d485b96ebb63/manager/0.log" Jul 06 02:12:02.439709 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.439682 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_012564a6-7f9e-4886-a4e3-9b9d699b79a6/alertmanager/0.log" Jul 06 02:12:02.465907 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.465850 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_012564a6-7f9e-4886-a4e3-9b9d699b79a6/config-reloader/0.log" Jul 06 02:12:02.486695 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.486665 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_012564a6-7f9e-4886-a4e3-9b9d699b79a6/kube-rbac-proxy-web/0.log" Jul 06 02:12:02.511268 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.511248 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_012564a6-7f9e-4886-a4e3-9b9d699b79a6/kube-rbac-proxy/0.log" Jul 06 02:12:02.535995 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.535974 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_012564a6-7f9e-4886-a4e3-9b9d699b79a6/kube-rbac-proxy-metric/0.log" Jul 06 02:12:02.558919 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.558827 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_012564a6-7f9e-4886-a4e3-9b9d699b79a6/prom-label-proxy/0.log" Jul 06 02:12:02.582320 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.582297 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_012564a6-7f9e-4886-a4e3-9b9d699b79a6/init-config-reloader/0.log" Jul 06 02:12:02.654004 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.653971 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-7xpnt_cc3ffe3a-2925-4815-899a-0831d4ab8135/kube-state-metrics/0.log" Jul 06 02:12:02.673767 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.673736 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-7xpnt_cc3ffe3a-2925-4815-899a-0831d4ab8135/kube-rbac-proxy-main/0.log" Jul 06 02:12:02.696204 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.696184 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-7xpnt_cc3ffe3a-2925-4815-899a-0831d4ab8135/kube-rbac-proxy-self/0.log" Jul 06 02:12:02.859185 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.859152 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjngt_d0580fae-7b59-4076-8312-635b9daf139f/node-exporter/0.log" Jul 06 02:12:02.880665 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.880631 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjngt_d0580fae-7b59-4076-8312-635b9daf139f/kube-rbac-proxy/0.log" Jul 06 02:12:02.906008 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:02.905990 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjngt_d0580fae-7b59-4076-8312-635b9daf139f/init-textfile/0.log" Jul 06 02:12:03.010176 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.010152 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fmwbw_646b659c-6b5c-4bf1-859b-3dd93a755697/kube-rbac-proxy-main/0.log" Jul 06 02:12:03.030823 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.030802 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fmwbw_646b659c-6b5c-4bf1-859b-3dd93a755697/kube-rbac-proxy-self/0.log" Jul 06 02:12:03.053948 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.053923 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-fmwbw_646b659c-6b5c-4bf1-859b-3dd93a755697/openshift-state-metrics/0.log" Jul 06 02:12:03.326898 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.326850 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f8479dc67-h8wt8_591d82c6-fa27-40a3-8757-72e37e9e13f4/telemeter-client/0.log" Jul 06 02:12:03.348521 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.348500 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f8479dc67-h8wt8_591d82c6-fa27-40a3-8757-72e37e9e13f4/reload/0.log" Jul 06 02:12:03.369420 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.369401 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f8479dc67-h8wt8_591d82c6-fa27-40a3-8757-72e37e9e13f4/kube-rbac-proxy/0.log" Jul 06 02:12:03.401833 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.401808 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fcfd97664-99kxx_e885ed9c-32c5-4047-9217-9db6832fc046/thanos-query/0.log" Jul 06 02:12:03.425176 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.425157 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fcfd97664-99kxx_e885ed9c-32c5-4047-9217-9db6832fc046/kube-rbac-proxy-web/0.log" Jul 06 02:12:03.446944 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.446927 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fcfd97664-99kxx_e885ed9c-32c5-4047-9217-9db6832fc046/kube-rbac-proxy/0.log" Jul 06 02:12:03.470598 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.470575 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fcfd97664-99kxx_e885ed9c-32c5-4047-9217-9db6832fc046/prom-label-proxy/0.log" Jul 06 02:12:03.491315 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.491293 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fcfd97664-99kxx_e885ed9c-32c5-4047-9217-9db6832fc046/kube-rbac-proxy-rules/0.log" Jul 06 02:12:03.511738 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:03.511708 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-fcfd97664-99kxx_e885ed9c-32c5-4047-9217-9db6832fc046/kube-rbac-proxy-metrics/0.log" Jul 06 02:12:05.328848 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.327585 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/1.log" Jul 06 02:12:05.338810 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.338763 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-hvddn_5cf6d3dc-566f-4abf-9165-59525dcaefa7/console-operator/2.log" Jul 06 02:12:05.630194 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630115 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt"] Jul 06 02:12:05.630563 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630550 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" Jul 06 02:12:05.630610 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630564 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" Jul 06 02:12:05.630610 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630573 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" Jul 06 02:12:05.630610 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630578 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" Jul 06 02:12:05.630610 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630607 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" Jul 06 02:12:05.630732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630613 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" Jul 06 02:12:05.630732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630621 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" Jul 06 02:12:05.630732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630626 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" Jul 06 02:12:05.630732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630703 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="042bc62b-0da9-4986-8903-462a941a477f" containerName="istio-proxy" Jul 06 02:12:05.630732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630713 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e52f005-3b02-4245-8cea-fb07acac97c2" containerName="istio-proxy" Jul 06 02:12:05.630732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630718 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" Jul 06 02:12:05.630732 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.630725 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca8e0225-d834-4ddc-bc29-ac680d9d3bc4" containerName="istio-proxy" Jul 06 02:12:05.634102 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.634087 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.635980 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.635958 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdvmn\"/\"kube-root-ca.crt\"" Jul 06 02:12:05.636083 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.635958 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdvmn\"/\"openshift-service-ca.crt\"" Jul 06 02:12:05.636083 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.636005 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jdvmn\"/\"default-dockercfg-tg2sc\"" Jul 06 02:12:05.640956 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.640931 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt"] Jul 06 02:12:05.694935 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.694910 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-sys\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.695074 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.694938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-podres\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.695074 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.694994 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-proc\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.695074 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.695034 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmtv6\" (UniqueName: \"kubernetes.io/projected/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-kube-api-access-fmtv6\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.695074 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.695072 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-lib-modules\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795595 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795565 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-proc\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795730 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795608 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmtv6\" (UniqueName: \"kubernetes.io/projected/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-kube-api-access-fmtv6\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795730 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795654 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-lib-modules\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795730 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795675 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-sys\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795730 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795694 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-podres\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795730 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795693 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-proc\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795950 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795768 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-sys\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795950 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795806 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-podres\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.795950 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.795806 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-lib-modules\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.803360 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.803339 2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmtv6\" (UniqueName: \"kubernetes.io/projected/8d3e79fe-5fd3-4517-84f9-528fc1d085fc-kube-api-access-fmtv6\") pod \"perf-node-gather-daemonset-lvnvt\" (UID: \"8d3e79fe-5fd3-4517-84f9-528fc1d085fc\") " pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:05.908589 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.908513 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d879dd9fb-xf54x_e9cb887b-1da4-4f0e-add3-7b239286f4e5/console/0.log" Jul 06 02:12:05.946967 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:05.946935 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:06.086656 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:06.086628 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt"] Jul 06 02:12:06.088561 ip-10-0-133-193 kubenswrapper[2574]: W0706 02:12:06.088534 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8d3e79fe_5fd3_4517_84f9_528fc1d085fc.slice/crio-a8a82a88dfd93e964946e679680a575182f30b3746703380ff15a33eb9cf2bb2 WatchSource:0}: Error finding container a8a82a88dfd93e964946e679680a575182f30b3746703380ff15a33eb9cf2bb2: Status 404 returned error can't find the container with id a8a82a88dfd93e964946e679680a575182f30b3746703380ff15a33eb9cf2bb2 Jul 06 02:12:06.228816 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:06.228782 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" event={"ID":"8d3e79fe-5fd3-4517-84f9-528fc1d085fc","Type":"ContainerStarted","Data":"09b678e37fc6579a24fad0abc43f7e510aaae8928c51ef1059e98d6d22e5e68d"} Jul 06 02:12:06.228995 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:06.228839 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" event={"ID":"8d3e79fe-5fd3-4517-84f9-528fc1d085fc","Type":"ContainerStarted","Data":"a8a82a88dfd93e964946e679680a575182f30b3746703380ff15a33eb9cf2bb2"} Jul 06 02:12:06.228995 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:06.228909 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:06.243755 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:06.243712 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" podStartSLOduration=1.24369767 podStartE2EDuration="1.24369767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 02:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 02:12:06.242578574 +0000 UTC m=+2724.377869080" watchObservedRunningTime="2026-07-06 02:12:06.24369767 +0000 UTC m=+2724.378988174" Jul 06 02:12:07.632468 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:07.632438 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dmkmq_b07fe06c-ab11-4f51-8a99-286d02bad557/dns/0.log" Jul 06 02:12:07.652433 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:07.652414 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dmkmq_b07fe06c-ab11-4f51-8a99-286d02bad557/kube-rbac-proxy/0.log" Jul 06 02:12:07.721539 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:07.721504 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6x7cb_ca72464b-98ac-4773-adec-7a0df6508fa7/dns-node-resolver/0.log" Jul 06 02:12:08.344829 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:08.344798 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9hcgl_b1a91258-14a4-45b3-ba6c-41ec9b31b731/node-ca/0.log" Jul 06 02:12:09.308898 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:09.308843 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46nlc7_b11c570f-2368-42f1-a601-af6747c34507/istio-proxy/0.log" Jul 06 02:12:09.415214 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:09.415184 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-stbx7_478f87cf-c161-4a86-b3dd-2d71dbf45de3/discovery/0.log" Jul 06 02:12:09.459073 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:09.459048 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f644fb7f6-ldnm4_e570dbbc-7813-4a2b-b689-a208edf74ebc/kube-auth-proxy/0.log" Jul 06 02:12:09.485801 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:09.485777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8wf7_abde296b-bf0d-45e4-8db4-d79ee2e60e0f/istio-proxy/0.log" Jul 06 02:12:09.511686 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:09.511660 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8wf7_abde296b-bf0d-45e4-8db4-d79ee2e60e0f/istio-proxy/1.log" Jul 06 02:12:09.540186 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:09.540162 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z9qkm_b07852b5-6abf-422e-a7d0-699270afbb06/payload-pre-processing/0.log" Jul 06 02:12:09.619806 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:09.619764 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7995fbff9c-klmcz_5fe2ae0a-62b0-4938-a271-bb810c1d533c/router/0.log" Jul 06 02:12:10.291973 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:10.291949 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4q7d7_2e0ee386-6ef9-4c02-8d0d-5cb81a1b2832/serve-healthcheck-canary/0.log" Jul 06 02:12:10.808162 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:10.808130 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-vtffd_066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc/insights-operator/0.log" Jul 06 02:12:10.808548 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:10.808321 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-vtffd_066fe0eb-bed2-44c0-b4b4-bc80a71b1dbc/insights-operator/1.log" Jul 06 02:12:10.893632 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:10.893610 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tllrv_f334001b-d1e4-4107-b5c8-1412e573b2aa/kube-rbac-proxy/0.log" Jul 06 02:12:10.913636 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:10.913611 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tllrv_f334001b-d1e4-4107-b5c8-1412e573b2aa/exporter/0.log" Jul 06 02:12:10.934287 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:10.934261 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tllrv_f334001b-d1e4-4107-b5c8-1412e573b2aa/extractor/0.log" Jul 06 02:12:12.245216 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:12.245191 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jdvmn/perf-node-gather-daemonset-lvnvt" Jul 06 02:12:13.323131 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:13.323100 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-f669l_3e7ae9b0-50f2-48f3-b07c-d5d58badcee7/manager/0.log" Jul 06 02:12:13.357771 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:13.357742 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6ddd7f5dc5-b24gf_ea7c9960-615e-4a5f-9011-3377d60cdd14/maas-api/0.log" Jul 06 02:12:13.399887 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:13.399839 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29721720-5v9h2_be3e159d-d3c4-4954-bb8b-0ee8fa05719e/cleanup/0.log" Jul 06 02:12:13.500148 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:13.500115 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6d6f449ff5-jszv6_951492ed-9116-467d-ab29-d0cb52e555dd/manager/0.log" Jul 06 02:12:13.783760 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:13.783679 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656987b587-lx7xt_5439a382-34f1-4d8f-bdf3-48db422e2539/manager/0.log" Jul 06 02:12:13.805595 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:13.805572 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656987b587-lx7xt_5439a382-34f1-4d8f-bdf3-48db422e2539/copy-manifests/0.log" Jul 06 02:12:13.873905 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:13.873876 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-8m2q9_433ef9c0-615f-4cd4-ba2c-91b3bdefc0c9/postgres/0.log" Jul 06 02:12:16.200493 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:16.200462 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46nlc7_b11c570f-2368-42f1-a601-af6747c34507/istio-proxy/0.log" Jul 06 02:12:16.304228 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:16.304193 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-stbx7_478f87cf-c161-4a86-b3dd-2d71dbf45de3/discovery/0.log" Jul 06 02:12:16.332326 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:16.332294 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f644fb7f6-ldnm4_e570dbbc-7813-4a2b-b689-a208edf74ebc/kube-auth-proxy/0.log" Jul 06 02:12:16.349984 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:16.349957 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8wf7_abde296b-bf0d-45e4-8db4-d79ee2e60e0f/istio-proxy/0.log" Jul 06 02:12:16.380106 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:16.380062 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8wf7_abde296b-bf0d-45e4-8db4-d79ee2e60e0f/istio-proxy/1.log" Jul 06 02:12:16.399733 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:16.399707 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z9qkm_b07852b5-6abf-422e-a7d0-699270afbb06/payload-pre-processing/0.log" Jul 06 02:12:16.465461 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:16.465383 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7995fbff9c-klmcz_5fe2ae0a-62b0-4938-a271-bb810c1d533c/router/0.log" Jul 06 02:12:17.714275 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:17.714248 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46nlc7_b11c570f-2368-42f1-a601-af6747c34507/istio-proxy/0.log" Jul 06 02:12:17.816222 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:17.816141 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-stbx7_478f87cf-c161-4a86-b3dd-2d71dbf45de3/discovery/0.log" Jul 06 02:12:17.847902 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:17.847873 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f644fb7f6-ldnm4_e570dbbc-7813-4a2b-b689-a208edf74ebc/kube-auth-proxy/0.log" Jul 06 02:12:17.869517 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:17.869490 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8wf7_abde296b-bf0d-45e4-8db4-d79ee2e60e0f/istio-proxy/0.log" Jul 06 02:12:17.897219 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:17.897194 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-z8wf7_abde296b-bf0d-45e4-8db4-d79ee2e60e0f/istio-proxy/1.log" Jul 06 02:12:17.917299 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:17.917273 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-z9qkm_b07852b5-6abf-422e-a7d0-699270afbb06/payload-pre-processing/0.log" Jul 06 02:12:17.975489 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:17.975464 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7995fbff9c-klmcz_5fe2ae0a-62b0-4938-a271-bb810c1d533c/router/0.log" Jul 06 02:12:22.206112 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.206085 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lpd78_9eab25ce-e60c-439c-8c4c-befdee9c1be3/kube-multus-additional-cni-plugins/0.log" Jul 06 02:12:22.226365 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.226340 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lpd78_9eab25ce-e60c-439c-8c4c-befdee9c1be3/egress-router-binary-copy/0.log" Jul 06 02:12:22.249929 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.249908 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lpd78_9eab25ce-e60c-439c-8c4c-befdee9c1be3/cni-plugins/0.log" Jul 06 02:12:22.270519 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.270499 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lpd78_9eab25ce-e60c-439c-8c4c-befdee9c1be3/bond-cni-plugin/0.log" Jul 06 02:12:22.292806 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.292783 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lpd78_9eab25ce-e60c-439c-8c4c-befdee9c1be3/routeoverride-cni/0.log" Jul 06 02:12:22.312847 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.312825 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lpd78_9eab25ce-e60c-439c-8c4c-befdee9c1be3/whereabouts-cni-bincopy/0.log" Jul 06 02:12:22.332640 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.332622 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lpd78_9eab25ce-e60c-439c-8c4c-befdee9c1be3/whereabouts-cni/0.log" Jul 06 02:12:22.367225 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.367200 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qts96_7e89584c-ddcf-48db-9695-696f9b47dea3/kube-multus/0.log" Jul 06 02:12:22.458487 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.458407 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cn8h2_b0a348e4-0203-4974-b406-002dc6a05d58/network-metrics-daemon/0.log" Jul 06 02:12:22.476808 ip-10-0-133-193 kubenswrapper[2574]: I0706 02:12:22.476782 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cn8h2_b0a348e4-0203-4974-b406-002dc6a05d58/kube-rbac-proxy/0.log"