Jul 14 13:54:20.652605 ip-10-0-142-141 systemd[1]: Starting Kubernetes Kubelet... Jul 14 13:54:21.105258 ip-10-0-142-141 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:54:21.105258 ip-10-0-142-141 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 13:54:21.105258 ip-10-0-142-141 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:54:21.105258 ip-10-0-142-141 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 13:54:21.105258 ip-10-0-142-141 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 13:54:21.107600 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.107469 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 13:54:21.113880 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113861 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:54:21.113880 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113879 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113885 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113888 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113892 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113896 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113899 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113902 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113905 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113908 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113911 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113914 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113917 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113920 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113922 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113927 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113931 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113935 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113938 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113940 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113943 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:54:21.113950 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113953 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113956 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113959 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113962 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113965 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113968 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113972 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113975 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113978 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113980 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113983 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113986 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113988 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113991 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113994 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113996 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.113999 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114002 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114005 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114007 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:54:21.114492 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114010 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114013 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114016 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114018 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114021 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114024 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114026 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114029 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114032 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114034 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114037 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114041 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114044 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114047 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114053 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114058 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114061 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114064 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114067 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114071 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:54:21.115028 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114074 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114078 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114080 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114083 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114086 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114089 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114092 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114094 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114097 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114100 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114103 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114105 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114108 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114111 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114114 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114118 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114121 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114124 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114127 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114129 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:54:21.115562 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114132 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114135 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114138 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114140 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114143 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114146 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114150 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114153 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114156 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114159 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114161 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114165 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114169 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114172 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114175 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114178 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114181 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114184 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114984 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:54:21.116058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114992 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114995 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.114999 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115002 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115005 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115008 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115011 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115014 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115017 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115020 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115023 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115026 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115028 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115031 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115034 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115038 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115041 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115044 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115047 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115051 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:54:21.116608 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115062 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115065 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115068 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115071 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115074 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115077 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115080 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115082 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115085 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115088 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115091 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115095 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115099 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115103 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115106 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115110 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115113 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115115 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115118 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:54:21.117175 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115121 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115124 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115127 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115130 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115133 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115136 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115138 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115141 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115144 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115146 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115149 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115152 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115156 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115159 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115165 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115168 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115171 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115174 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115177 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115179 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:54:21.117668 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115182 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115185 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115187 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115190 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115193 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115196 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115198 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115202 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115204 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115207 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115210 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115212 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115215 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115218 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115221 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115223 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115226 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115228 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115233 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:54:21.118190 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115237 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115240 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115243 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115246 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115249 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115252 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115255 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115258 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115261 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115264 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115267 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115270 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115273 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115275 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115278 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115281 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115284 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115286 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115289 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115292 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:54:21.118696 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115743 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115750 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115754 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115757 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115760 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115763 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115766 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115769 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115772 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115775 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115778 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115781 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115784 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115786 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115789 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115792 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115794 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115798 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115801 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115804 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:54:21.119223 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115807 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115810 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115813 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115816 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115819 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115821 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115824 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115827 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115830 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115833 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115836 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115838 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115841 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115844 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115846 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115849 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115852 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115854 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115857 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115860 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:54:21.119769 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115863 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115866 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115869 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115872 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115874 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115877 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115880 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115883 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115886 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115889 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115892 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115895 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115898 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115901 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115904 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115906 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115910 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115912 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115915 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115918 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:54:21.120409 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115920 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115923 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115926 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115928 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115931 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115935 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115938 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115941 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115943 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115946 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115949 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115951 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115954 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115957 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115959 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115962 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115965 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115967 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115972 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:54:21.120933 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115977 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115980 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115984 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115987 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115990 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115994 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.115999 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116002 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116005 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116007 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116010 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116013 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116016 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116019 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116021 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116024 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116027 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116031 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116034 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:54:21.121451 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116038 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116118 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116130 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116146 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116152 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116156 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116160 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116164 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116169 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116172 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116175 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116179 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116183 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116186 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116190 2564 flags.go:64] FLAG: --cgroup-root="" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116193 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116196 2564 flags.go:64] FLAG: --client-ca-file="" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116200 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116203 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116208 2564 flags.go:64] FLAG: --cluster-domain="" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116212 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116216 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116219 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116223 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 13:54:21.121954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116232 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116235 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116238 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116243 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116246 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116250 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116256 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116259 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116263 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116267 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116271 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116274 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116277 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116280 2564 flags.go:64] FLAG: --enable-server="true" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116284 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116288 2564 flags.go:64] FLAG: --event-burst="100" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116292 2564 flags.go:64] FLAG: --event-qps="50" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116295 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116298 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116301 2564 flags.go:64] FLAG: --eviction-hard="" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116305 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116308 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116311 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116315 2564 flags.go:64] FLAG: --eviction-soft="" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116318 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 13:54:21.122629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116321 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116324 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116327 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116330 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116333 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116338 2564 flags.go:64] FLAG: --feature-gates="" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116343 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116346 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116349 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116365 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116369 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116373 2564 flags.go:64] FLAG: --help="false" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116376 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116381 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116384 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116387 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116391 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116395 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116398 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116401 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116404 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116407 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116410 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116414 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 13:54:21.123320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116417 2564 flags.go:64] FLAG: --kube-reserved="" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116420 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116423 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116426 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116429 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116432 2564 flags.go:64] FLAG: --lock-file="" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116435 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116438 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116442 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116447 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116451 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116454 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116458 2564 flags.go:64] FLAG: --logging-format="text" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116462 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116467 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116470 2564 flags.go:64] FLAG: --manifest-url="" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116473 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116478 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116481 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116485 2564 flags.go:64] FLAG: --max-pods="110" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116489 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116493 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116496 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116500 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116503 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 13:54:21.123977 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116506 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116509 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116517 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116520 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116523 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116527 2564 flags.go:64] FLAG: --pod-cidr="" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116530 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116537 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116540 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116543 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116546 2564 flags.go:64] FLAG: --port="10250" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116550 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116553 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fde330047c934377" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116556 2564 flags.go:64] FLAG: --qos-reserved="" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116559 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116563 2564 flags.go:64] FLAG: --register-node="true" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116566 2564 flags.go:64] FLAG: --register-with-taints="" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116570 2564 flags.go:64] FLAG: --registry-burst="10" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116573 2564 flags.go:64] FLAG: --registry-qps="5" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116576 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116582 2564 flags.go:64] FLAG: --reserved-memory="" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116586 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116590 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116593 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116597 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 13:54:21.124727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116600 2564 flags.go:64] FLAG: --runonce="false" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116603 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116606 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116609 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116614 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116618 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116621 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116625 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116628 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116632 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116635 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116638 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116641 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116644 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116648 2564 flags.go:64] FLAG: --system-cgroups="" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116651 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116657 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116660 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116663 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116668 2564 flags.go:64] FLAG: --tls-min-version="" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116671 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116674 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116677 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116680 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116683 2564 flags.go:64] FLAG: --v="2" Jul 14 13:54:21.125684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116687 2564 flags.go:64] FLAG: --version="false" Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116691 2564 flags.go:64] FLAG: --vmodule="" Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116696 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.116699 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116826 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116830 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116834 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116838 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116842 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116845 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116849 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116855 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116858 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116861 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116863 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116866 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116869 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116872 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116875 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116878 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:54:21.126502 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116881 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116884 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116887 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116890 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116893 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116895 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116899 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116902 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116904 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116907 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116910 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116913 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116916 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116918 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116922 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116925 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116927 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116930 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116934 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116936 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:54:21.127037 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116939 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116942 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116945 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116949 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116952 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116955 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116958 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116962 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116965 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116969 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116971 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116974 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116977 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116980 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116983 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116985 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116989 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116992 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116995 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.116998 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:54:21.127619 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117001 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117003 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117006 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117009 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117011 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117014 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117017 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117020 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117023 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117026 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117028 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117031 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117034 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117037 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117039 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117045 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117049 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117052 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117055 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:54:21.128134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117058 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117061 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117064 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117067 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117070 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117073 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117084 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117088 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117091 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117095 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117097 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117100 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117103 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117106 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117109 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117112 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117115 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117118 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117120 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117123 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:54:21.128733 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117126 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117129 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117132 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.117134 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.117832 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.127294 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.127316 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.127338 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.127344 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127427 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127433 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127437 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127440 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127443 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127446 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:54:21.129242 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127450 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127453 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127455 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127458 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127461 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127464 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127467 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127470 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127472 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127475 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127478 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127481 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127484 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127486 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127489 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127492 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127495 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127499 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127503 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127506 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:54:21.129660 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127509 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127513 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127522 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127525 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127528 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127531 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127534 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127537 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127540 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127542 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127545 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127548 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127551 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127554 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127557 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127559 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127562 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127565 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127568 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127571 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:54:21.130189 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127575 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127577 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127580 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127583 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127586 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127589 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127591 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127595 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127598 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127600 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127603 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127608 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127611 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127616 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127619 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127623 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127626 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127629 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127633 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:54:21.130728 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127636 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127639 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127643 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127645 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127648 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127651 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127654 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127657 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127659 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127662 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127666 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127670 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127674 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127676 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127680 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127683 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127686 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127689 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127708 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127712 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:54:21.131215 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127715 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127719 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127723 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127726 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127729 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127732 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127736 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127739 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127743 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127746 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127749 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127752 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127755 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127758 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.127764 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:54:21.131759 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127892 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127898 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127902 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127905 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127909 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127912 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127915 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127919 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127921 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127924 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127927 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127930 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127934 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127937 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127939 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127942 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127945 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127948 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127952 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 13:54:21.132157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127957 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127960 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127963 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127966 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127969 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127972 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127976 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127979 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127982 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127985 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127988 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127991 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127994 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127997 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.127999 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128002 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128005 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128008 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128011 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128014 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 13:54:21.132666 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128017 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128019 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128022 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128025 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128027 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128030 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128034 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128038 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128040 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128043 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128046 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128048 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128051 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128054 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128057 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128059 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128062 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128065 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128068 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 13:54:21.133187 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128071 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128074 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128078 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128082 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128085 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128088 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128091 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128093 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128097 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128099 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128102 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128105 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128108 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128111 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128113 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128116 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128119 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128122 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128124 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 13:54:21.133748 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128127 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128131 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128133 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128136 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128139 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128142 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128145 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128148 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128150 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128153 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128156 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128159 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128162 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128165 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128168 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128170 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128173 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128176 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128179 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128182 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128184 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 13:54:21.134230 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:21.128187 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 13:54:21.134787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.128192 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 13:54:21.134787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.128322 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 13:54:21.134787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.130842 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 13:54:21.134787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.132467 2564 server.go:1019] "Starting client certificate rotation" Jul 14 13:54:21.134787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.132573 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 13:54:21.134787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.132615 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 13:54:21.135274 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.135260 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 13:54:21.136508 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.136487 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 13:54:21.143098 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.143072 2564 log.go:25] "Validated CRI v1 runtime API" Jul 14 13:54:21.145186 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.145170 2564 log.go:25] "Validated CRI v1 image API" Jul 14 13:54:21.146613 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.146596 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 13:54:21.150233 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.150212 2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c097cc44-98bb-4431-b699-79009ae7bf74:/dev/nvme0n1p4 ed04861a-4529-49bc-a090-83960310e90c:/dev/nvme0n1p3] Jul 14 13:54:21.150305 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.150232 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 13:54:21.156548 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.156423 2564 manager.go:216] Machine: {Timestamp:2026-07-14 13:54:21.154950843 +0000 UTC m=+0.381633753 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099181 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22d08500b0141bb6c275e36c205aba SystemUUID:ec22d085-00b0-141b-b6c2-75e36c205aba BootID:efad5665-f6d5-4eaa-b6d7-60db0b639a1b Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b2:8f:5a:9b:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b2:8f:5a:9b:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:61:37:5e:78:e8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 13:54:21.156548 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.156535 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 13:54:21.156689 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.156631 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 13:54:21.160143 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.160118 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 13:54:21.160288 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.160147 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 13:54:21.160920 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.160909 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 13:54:21.160964 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.160924 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 13:54:21.160964 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.160931 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 13:54:21.160964 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.160948 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 13:54:21.162269 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.162258 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 13:54:21.165545 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.165533 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 14 13:54:21.165684 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.165673 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 13:54:21.166428 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.166417 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 14 13:54:21.166486 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.166442 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 13:54:21.166486 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.166459 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 13:54:21.166486 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.166470 2564 kubelet.go:398] "Adding apiserver pod source" Jul 14 13:54:21.166486 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.166481 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 13:54:21.166943 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.166931 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 13:54:21.166978 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.166952 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 13:54:21.168778 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.168758 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 13:54:21.169761 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.169742 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 13:54:21.171851 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.171836 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 13:54:21.171917 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.171870 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 13:54:21.172034 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172023 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 13:54:21.172067 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172039 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 13:54:21.172067 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172045 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 13:54:21.172067 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172051 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 13:54:21.172067 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172060 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 13:54:21.172067 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172066 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 13:54:21.172744 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172734 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 13:54:21.172744 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172744 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 13:54:21.172810 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172752 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 13:54:21.172810 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172758 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 13:54:21.172810 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172768 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 13:54:21.172810 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.172777 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 13:54:21.173760 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.173750 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 13:54:21.173803 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.173761 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 13:54:21.177229 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.177212 2564 server.go:1268] "Started kubelet" Jul 14 13:54:21.177446 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.177412 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 13:54:21.178349 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.178288 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 13:54:21.178448 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.178372 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 13:54:21.179563 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.179541 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 13:54:21.179601 ip-10-0-142-141 systemd[1]: Started Kubernetes Kubelet. Jul 14 13:54:21.180716 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.180701 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 14 13:54:21.181311 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.181261 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 13:54:21.181401 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.181323 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 13:54:21.181454 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.181401 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 13:54:21.182408 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.182373 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 13:54:21.185309 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.184251 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-141.ec2.internal.18c22c6fd286d5c8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-141.ec2.internal,UID:ip-10-0-142-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-141.ec2.internal,},FirstTimestamp:2026-07-14 13:54:21.17718164 +0000 UTC m=+0.403864551,LastTimestamp:2026-07-14 13:54:21.17718164 +0000 UTC m=+0.403864551,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-141.ec2.internal,}" Jul 14 13:54:21.185681 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.185660 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 13:54:21.185767 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.185666 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 13:54:21.185767 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.185729 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:21.186313 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.186295 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 13:54:21.186313 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.186316 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 13:54:21.186469 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.186371 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 13:54:21.186526 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.186482 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 14 13:54:21.186526 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.186491 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 14 13:54:21.186701 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.186685 2564 factory.go:55] Registering systemd factory Jul 14 13:54:21.186779 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.186704 2564 factory.go:223] Registration of the systemd container factory successfully Jul 14 13:54:21.187190 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.187169 2564 factory.go:153] Registering CRI-O factory Jul 14 13:54:21.187271 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.187195 2564 factory.go:223] Registration of the crio container factory successfully Jul 14 13:54:21.187271 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.187260 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 13:54:21.187423 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.187317 2564 factory.go:103] Registering Raw factory Jul 14 13:54:21.187423 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.187351 2564 manager.go:1196] Started watching for new ooms in manager Jul 14 13:54:21.187946 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.187926 2564 manager.go:349] Starting recovery of all containers Jul 14 13:54:21.192288 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.192256 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 13:54:21.192467 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.192437 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 13:54:21.195586 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.195563 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rbnjj" Jul 14 13:54:21.201038 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.201017 2564 manager.go:354] Recovery completed Jul 14 13:54:21.202532 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.202500 2564 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 14 13:54:21.204538 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.204521 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rbnjj" Jul 14 13:54:21.205781 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.205768 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:54:21.207852 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.207834 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:54:21.207945 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.207870 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:54:21.207945 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.207887 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:54:21.208710 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.208576 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 13:54:21.208795 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.208715 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 13:54:21.208795 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.208735 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 14 13:54:21.208795 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.208584 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:54:21.209783 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.209764 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:54:21.209858 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.209801 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:54:21.209858 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.209820 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:54:21.209858 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.209826 2564 policy_none.go:49] "None policy: Start" Jul 14 13:54:21.209858 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.209844 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 13:54:21.210050 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.209859 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 13:54:21.210465 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.210391 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-141.ec2.internal.18c22c6fd45ad3b2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-141.ec2.internal,UID:ip-10-0-142-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-141.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-141.ec2.internal,},FirstTimestamp:2026-07-14 13:54:21.207851954 +0000 UTC m=+0.434534867,LastTimestamp:2026-07-14 13:54:21.207851954 +0000 UTC m=+0.434534867,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-141.ec2.internal,}" Jul 14 13:54:21.211043 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.211030 2564 policy_none.go:47] "Start" Jul 14 13:54:21.253065 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.253047 2564 manager.go:337] "Starting Device Plugin manager" Jul 14 13:54:21.253188 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.253083 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 13:54:21.253188 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.253094 2564 server.go:87] "Starting device plugin registration server" Jul 14 13:54:21.253474 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.253459 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 13:54:21.253517 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.253478 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 13:54:21.253620 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.253602 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 13:54:21.253717 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.253689 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 13:54:21.253717 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.253699 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 13:54:21.254254 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.254232 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 13:54:21.254373 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.254288 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:21.354336 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.354279 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:54:21.355674 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.355634 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:54:21.355745 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.355675 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:54:21.355745 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.355686 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:54:21.355745 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.355714 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.362112 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.362082 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 13:54:21.362701 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.362678 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.362798 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.362716 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-141.ec2.internal\": node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:21.363595 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.363578 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 13:54:21.363651 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.363617 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 13:54:21.363651 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.363638 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 13:54:21.363718 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.363670 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 13:54:21.367448 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.367429 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:54:21.413949 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.413918 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:21.466618 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.466572 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal"] Jul 14 13:54:21.466788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.466663 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:54:21.468649 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.468630 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:54:21.468773 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.468667 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:54:21.468773 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.468682 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:54:21.470117 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.470103 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:54:21.470284 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.470270 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.470323 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.470318 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:54:21.470901 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.470884 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:54:21.470954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.470918 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:54:21.470954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.470930 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:54:21.471027 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.470884 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:54:21.471027 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.470997 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:54:21.471027 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.471011 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:54:21.472201 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.472182 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.472276 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.472225 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 13:54:21.472916 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.472901 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientMemory" Jul 14 13:54:21.472990 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.472934 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 13:54:21.472990 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.472949 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeHasSufficientPID" Jul 14 13:54:21.488581 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.488550 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e38e3cc78bf216faf085a021c57364a2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-141.ec2.internal\" (UID: \"e38e3cc78bf216faf085a021c57364a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.488675 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.488583 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26313c9496ac2ecad22f089933c11a34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal\" (UID: \"26313c9496ac2ecad22f089933c11a34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.488675 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.488601 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26313c9496ac2ecad22f089933c11a34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal\" (UID: \"26313c9496ac2ecad22f089933c11a34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.489941 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.489923 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-141.ec2.internal\" not found" node="ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.494020 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.494002 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-141.ec2.internal\" not found" node="ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.514136 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.514115 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:21.589055 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.589014 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e38e3cc78bf216faf085a021c57364a2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-141.ec2.internal\" (UID: \"e38e3cc78bf216faf085a021c57364a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.589055 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.589054 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26313c9496ac2ecad22f089933c11a34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal\" (UID: \"26313c9496ac2ecad22f089933c11a34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.589274 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.589073 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26313c9496ac2ecad22f089933c11a34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal\" (UID: \"26313c9496ac2ecad22f089933c11a34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.589274 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.589112 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26313c9496ac2ecad22f089933c11a34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal\" (UID: \"26313c9496ac2ecad22f089933c11a34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.589274 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.589117 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e38e3cc78bf216faf085a021c57364a2-config\") pod \"kube-apiserver-proxy-ip-10-0-142-141.ec2.internal\" (UID: \"e38e3cc78bf216faf085a021c57364a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.589274 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.589132 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26313c9496ac2ecad22f089933c11a34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal\" (UID: \"26313c9496ac2ecad22f089933c11a34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.614459 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.614379 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:21.716226 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.716180 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:21.792372 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.792340 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.797429 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:21.797397 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:21.816566 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.816533 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:21.918805 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:21.918701 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:22.018810 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:22.018775 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:22.119295 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:22.119238 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:22.131288 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.131257 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 13:54:22.131460 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.131437 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 13:54:22.207715 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.207638 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 13:49:21 +0000 UTC" deadline="2028-04-10 09:16:59.77454921 +0000 UTC" Jul 14 13:54:22.207715 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.207669 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15259h22m37.566884459s" Jul 14 13:54:22.219878 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:22.219844 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-141.ec2.internal\" not found" Jul 14 13:54:22.232087 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.232058 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:54:22.286748 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.286715 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" Jul 14 13:54:22.301585 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.301560 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 13:54:22.303585 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.303571 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" Jul 14 13:54:22.315007 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.314982 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 13:54:22.331344 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:22.331295 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38e3cc78bf216faf085a021c57364a2.slice/crio-977a45c507e2a217c9c808d7a640151fe8c4c2736e71392e3e25c6c2e03690d9 WatchSource:0}: Error finding container 977a45c507e2a217c9c808d7a640151fe8c4c2736e71392e3e25c6c2e03690d9: Status 404 returned error can't find the container with id 977a45c507e2a217c9c808d7a640151fe8c4c2736e71392e3e25c6c2e03690d9 Jul 14 13:54:22.331548 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:22.331520 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26313c9496ac2ecad22f089933c11a34.slice/crio-26d93df380e670077eceaf18b02c790f93449ea5bfb97b710963eaf7478d04bd WatchSource:0}: Error finding container 26d93df380e670077eceaf18b02c790f93449ea5bfb97b710963eaf7478d04bd: Status 404 returned error can't find the container with id 26d93df380e670077eceaf18b02c790f93449ea5bfb97b710963eaf7478d04bd Jul 14 13:54:22.336030 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.336012 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 13:54:22.367753 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.367687 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" event={"ID":"26313c9496ac2ecad22f089933c11a34","Type":"ContainerStarted","Data":"26d93df380e670077eceaf18b02c790f93449ea5bfb97b710963eaf7478d04bd"} Jul 14 13:54:22.368779 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.368755 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" event={"ID":"e38e3cc78bf216faf085a021c57364a2","Type":"ContainerStarted","Data":"977a45c507e2a217c9c808d7a640151fe8c4c2736e71392e3e25c6c2e03690d9"} Jul 14 13:54:22.430467 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.430441 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:54:22.549396 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:22.549315 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:54:23.168954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.168913 2564 apiserver.go:52] "Watching apiserver" Jul 14 13:54:23.178130 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.177918 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 13:54:23.178630 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.178447 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal","openshift-multus/multus-additional-cni-plugins-7nrrq","openshift-multus/multus-pf526","kube-system/global-pull-secret-syncer-w8zzc","kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8","openshift-dns/node-resolver-9nmxt","openshift-multus/network-metrics-daemon-zrcm7","openshift-network-diagnostics/network-check-target-gz22v","openshift-network-operator/iptables-alerter-89cc4","openshift-ovn-kubernetes/ovnkube-node-kspwb","kube-system/konnectivity-agent-cphc7","openshift-cluster-node-tuning-operator/tuned-ftx77","openshift-image-registry/node-ca-lp6xp"] Jul 14 13:54:23.180798 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.180149 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.181663 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.181639 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.183286 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.183265 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 13:54:23.183588 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.183571 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5q2nw\"" Jul 14 13:54:23.183962 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.183943 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pf526" Jul 14 13:54:23.184163 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.184141 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 13:54:23.185218 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.185198 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.186395 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.187454 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.189624 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.189815 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.189891 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.190138 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6wc74\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.190144 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dfmvs\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.190331 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.190413 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.190431 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:23.190571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.190485 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 13:54:23.191174 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.190653 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 13:54:23.191174 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.190730 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 13:54:23.191174 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.190916 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 13:54:23.192947 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.191650 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.193932 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.193434 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.194058 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.194036 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 13:54:23.194416 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.194348 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nsq8f\"" Jul 14 13:54:23.194770 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.194552 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 13:54:23.194770 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.194612 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 13:54:23.195124 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.194988 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:23.195124 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.195085 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:23.196833 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.196814 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 13:54:23.197320 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.197304 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 13:54:23.197744 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.197729 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2jzxx\"" Jul 14 13:54:23.198166 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.198139 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 13:54:23.198265 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.196904 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.202019 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.201996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-kubelet\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202114 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-systemd-units\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202114 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-slash\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202114 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-run-netns\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202114 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202110 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-systemd\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202137 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-ovnkube-config\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202188 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-env-overrides\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-hostroot\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4f6a09e-45c4-4875-908a-18f27e9c527c-dbus\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202262 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202287 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfe91ad7-93e0-48de-b5b5-58915fce187e-host\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202319 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-ovn\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202219 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.202819 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202653 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-cni-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.202819 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202690 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-cni-multus\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.202819 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202730 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cbjd\" (UniqueName: \"kubernetes.io/projected/dfe91ad7-93e0-48de-b5b5-58915fce187e-kube-api-access-9cbjd\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.202819 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202774 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-cni-netd\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.202819 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202806 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-ovnkube-script-lib\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202831 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvd9p\" (UniqueName: \"kubernetes.io/projected/6954d52c-d756-4559-8c23-5a25abc731f0-kube-api-access-qvd9p\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202855 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-netns\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202887 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/305e2065-0eb3-424c-b31f-50944b568790-hosts-file\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202912 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202937 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6954d52c-d756-4559-8c23-5a25abc731f0-ovn-node-metrics-cert\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202961 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-system-cni-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.202986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-socket-dir-parent\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203012 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-kubelet\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t55j5\" (UniqueName: \"kubernetes.io/projected/f9817318-85dd-4754-bc4e-c2222ae011f6-kube-api-access-t55j5\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203094 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/305e2065-0eb3-424c-b31f-50944b568790-tmp-dir\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203121 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw8qv\" (UniqueName: \"kubernetes.io/projected/305e2065-0eb3-424c-b31f-50944b568790-kube-api-access-jw8qv\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-etc-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203169 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-os-release\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203193 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9817318-85dd-4754-bc4e-c2222ae011f6-cni-binary-copy\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203237 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-conf-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203276 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-multus-certs\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203300 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4f6a09e-45c4-4875-908a-18f27e9c527c-kubelet-config\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203326 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-var-lib-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203375 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-node-log\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203488 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-cnibin\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203514 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-k8s-cni-cncf-io\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.203556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203544 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-daemon-config\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.204280 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203572 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-log-socket\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.204280 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203595 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-cni-bin\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.204280 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203625 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-cni-bin\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.204280 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203686 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-etc-kubernetes\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.204280 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.203712 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfe91ad7-93e0-48de-b5b5-58915fce187e-serviceca\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.204280 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.204195 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:54:23.204673 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.204651 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 13:54:23.204902 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.204883 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5gcwf\"" Jul 14 13:54:23.205157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.205136 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.205259 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.205241 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kqkvd\"" Jul 14 13:54:23.205506 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.205489 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 13:54:23.206145 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.205824 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 13:54:23.206520 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.206504 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:23.206790 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.206743 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:23.207855 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.207781 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:23.208405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.208210 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:54:23.208405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.208227 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 13:54:23.208405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.208277 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mcbsn\"" Jul 14 13:54:23.208405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.208385 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 13:54:23.208673 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.208447 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 13:54:23.210182 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.210159 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 13:54:23.210525 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.210507 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dw48j\"" Jul 14 13:54:23.210640 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.210576 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 13:54:23.227534 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.227511 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s7sj2" Jul 14 13:54:23.238460 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.238433 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s7sj2" Jul 14 13:54:23.288466 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.288439 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 13:54:23.304603 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304564 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/68a34d3e-b358-4ff4-88a5-bd3f69f33e55-agent-certs\") pod \"konnectivity-agent-cphc7\" (UID: \"68a34d3e-b358-4ff4-88a5-bd3f69f33e55\") " pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:23.304788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304618 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-cni-bin\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.304788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304648 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-etc-kubernetes\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.304788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-systemd\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.304788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-run\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.304788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304712 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-host\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.304788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-cni-bin\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.304788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-etc-kubernetes\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.304788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-slash\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304806 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-run-netns\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-sys-fs\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304777 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-slash\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304869 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304870 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-run-netns\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304920 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-systemd\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304959 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.304984 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-systemd\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305027 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4f6a09e-45c4-4875-908a-18f27e9c527c-dbus\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305067 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305121 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfe91ad7-93e0-48de-b5b5-58915fce187e-host\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305152 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dfe91ad7-93e0-48de-b5b5-58915fce187e-host\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.305195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305194 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-registration-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305239 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-var-lib-kubelet\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305252 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e4f6a09e-45c4-4875-908a-18f27e9c527c-dbus\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305275 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91084e9b-41a4-406a-8e99-40b269a9082f-tmp\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305319 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db21c734-4ec2-4cf8-9d3b-5a3095303e39-host-slash\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-ovn\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305376 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cbjd\" (UniqueName: \"kubernetes.io/projected/dfe91ad7-93e0-48de-b5b5-58915fce187e-kube-api-access-9cbjd\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305439 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvd9p\" (UniqueName: \"kubernetes.io/projected/6954d52c-d756-4559-8c23-5a25abc731f0-kube-api-access-qvd9p\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-netns\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305501 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-modprobe-d\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305549 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2dx2\" (UniqueName: \"kubernetes.io/projected/91084e9b-41a4-406a-8e99-40b269a9082f-kube-api-access-b2dx2\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305574 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305602 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305630 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/305e2065-0eb3-424c-b31f-50944b568790-tmp-dir\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305656 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw8qv\" (UniqueName: \"kubernetes.io/projected/305e2065-0eb3-424c-b31f-50944b568790-kube-api-access-jw8qv\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.305956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305682 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-etc-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-multus-certs\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-ovn\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305762 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4f6a09e-45c4-4875-908a-18f27e9c527c-kubelet-config\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305787 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysctl-conf\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cni-binary-copy\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305817 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-run-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305837 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db21c734-4ec2-4cf8-9d3b-5a3095303e39-iptables-alerter-script\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305863 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-node-log\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305889 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-cnibin\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-k8s-cni-cncf-io\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.305973 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-kubernetes\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306021 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-netns\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306058 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwfpr\" (UniqueName: \"kubernetes.io/projected/b4490358-1756-4a8e-b8a6-d2503b9fdf31-kube-api-access-nwfpr\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306091 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-log-socket\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306130 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-cni-bin\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.306834 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfe91ad7-93e0-48de-b5b5-58915fce187e-serviceca\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306182 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-socket-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/68a34d3e-b358-4ff4-88a5-bd3f69f33e55-konnectivity-ca\") pod \"konnectivity-agent-cphc7\" (UID: \"68a34d3e-b358-4ff4-88a5-bd3f69f33e55\") " pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306235 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-kubelet\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306260 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-systemd-units\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-ovnkube-config\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-env-overrides\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306334 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/305e2065-0eb3-424c-b31f-50944b568790-tmp-dir\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306361 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-multus-certs\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306401 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-hostroot\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306340 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-hostroot\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306440 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e4f6a09e-45c4-4875-908a-18f27e9c527c-kubelet-config\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91084e9b-41a4-406a-8e99-40b269a9082f-etc-tuned\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306482 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-node-log\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306510 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-cni-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306556 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-etc-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306589 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-cni-multus\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.307698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306596 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-cni-bin\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.306599 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306618 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306635 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-log-socket\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306645 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh8mk\" (UniqueName: \"kubernetes.io/projected/db21c734-4ec2-4cf8-9d3b-5a3095303e39-kube-api-access-wh8mk\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.306679 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret podName:e4f6a09e-45c4-4875-908a-18f27e9c527c nodeName:}" failed. No retries permitted until 2026-07-14 13:54:23.806649328 +0000 UTC m=+3.033332242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret") pod "global-pull-secret-syncer-w8zzc" (UID: "e4f6a09e-45c4-4875-908a-18f27e9c527c") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306709 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-cni-netd\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306559 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-run-k8s-cni-cncf-io\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306728 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-cni-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306771 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-ovnkube-script-lib\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306797 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-sys\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306845 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv7q8\" (UniqueName: \"kubernetes.io/projected/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-kube-api-access-dv7q8\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/305e2065-0eb3-424c-b31f-50944b568790-hosts-file\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306869 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dfe91ad7-93e0-48de-b5b5-58915fce187e-serviceca\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306880 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6954d52c-d756-4559-8c23-5a25abc731f0-ovn-node-metrics-cert\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306898 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-system-cni-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.308576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306925 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-cni-multus\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-system-cni-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306971 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-kubelet\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.306976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-cni-netd\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307276 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-socket-dir-parent\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307348 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-kubelet\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307011 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-systemd-units\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307636 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t55j5\" (UniqueName: \"kubernetes.io/projected/f9817318-85dd-4754-bc4e-c2222ae011f6-kube-api-access-t55j5\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307677 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysctl-d\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-lib-modules\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307742 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-os-release\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307764 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-ovnkube-script-lib\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307783 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9817318-85dd-4754-bc4e-c2222ae011f6-cni-binary-copy\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307886 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-conf-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307925 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cfb7\" (UniqueName: \"kubernetes.io/projected/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-kube-api-access-6cfb7\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-env-overrides\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307956 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-os-release\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.309427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.307991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-var-lib-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.308028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.308063 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-daemon-config\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.308091 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-device-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.308130 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysconfig\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.308164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-system-cni-dir\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.308537 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6954d52c-d756-4559-8c23-5a25abc731f0-ovnkube-config\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.308999 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.309065 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cnibin\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.309526 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-host-var-lib-kubelet\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.309634 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-host-run-ovn-kubernetes\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.309698 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-cnibin\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.309720 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f9817318-85dd-4754-bc4e-c2222ae011f6-cni-binary-copy\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.309791 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-conf-dir\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.309802 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-os-release\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.309952 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6954d52c-d756-4559-8c23-5a25abc731f0-var-lib-openvswitch\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.310007 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/305e2065-0eb3-424c-b31f-50944b568790-hosts-file\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.310105 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-socket-dir-parent\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.310258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.310186 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f9817318-85dd-4754-bc4e-c2222ae011f6-multus-daemon-config\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.317597 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.317566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvd9p\" (UniqueName: \"kubernetes.io/projected/6954d52c-d756-4559-8c23-5a25abc731f0-kube-api-access-qvd9p\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.318077 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.318025 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cbjd\" (UniqueName: \"kubernetes.io/projected/dfe91ad7-93e0-48de-b5b5-58915fce187e-kube-api-access-9cbjd\") pod \"node-ca-lp6xp\" (UID: \"dfe91ad7-93e0-48de-b5b5-58915fce187e\") " pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.318452 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.318423 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6954d52c-d756-4559-8c23-5a25abc731f0-ovn-node-metrics-cert\") pod \"ovnkube-node-kspwb\" (UID: \"6954d52c-d756-4559-8c23-5a25abc731f0\") " pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.318559 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.318499 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t55j5\" (UniqueName: \"kubernetes.io/projected/f9817318-85dd-4754-bc4e-c2222ae011f6-kube-api-access-t55j5\") pod \"multus-pf526\" (UID: \"f9817318-85dd-4754-bc4e-c2222ae011f6\") " pod="openshift-multus/multus-pf526" Jul 14 13:54:23.318559 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.318552 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw8qv\" (UniqueName: \"kubernetes.io/projected/305e2065-0eb3-424c-b31f-50944b568790-kube-api-access-jw8qv\") pod \"node-resolver-9nmxt\" (UID: \"305e2065-0eb3-424c-b31f-50944b568790\") " pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.409577 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409537 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:23.409577 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409579 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-kubernetes\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.409800 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409605 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwfpr\" (UniqueName: \"kubernetes.io/projected/b4490358-1756-4a8e-b8a6-d2503b9fdf31-kube-api-access-nwfpr\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:23.409800 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409631 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-socket-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.409800 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409653 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/68a34d3e-b358-4ff4-88a5-bd3f69f33e55-konnectivity-ca\") pod \"konnectivity-agent-cphc7\" (UID: \"68a34d3e-b358-4ff4-88a5-bd3f69f33e55\") " pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:23.409800 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-kubernetes\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.409800 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409689 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91084e9b-41a4-406a-8e99-40b269a9082f-etc-tuned\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.409800 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.409800 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409762 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh8mk\" (UniqueName: \"kubernetes.io/projected/db21c734-4ec2-4cf8-9d3b-5a3095303e39-kube-api-access-wh8mk\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.409800 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-sys\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409812 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409824 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-socket-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409895 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv7q8\" (UniqueName: \"kubernetes.io/projected/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-kube-api-access-dv7q8\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-sys\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysctl-d\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409963 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-lib-modules\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.409992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cfb7\" (UniqueName: \"kubernetes.io/projected/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-kube-api-access-6cfb7\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410022 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-os-release\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410050 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-device-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysconfig\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410099 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-system-cni-dir\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410144 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cnibin\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410157 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-system-cni-dir\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410172 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/68a34d3e-b358-4ff4-88a5-bd3f69f33e55-agent-certs\") pod \"konnectivity-agent-cphc7\" (UID: \"68a34d3e-b358-4ff4-88a5-bd3f69f33e55\") " pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:23.410210 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410201 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-systemd\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410225 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-run\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410248 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-host\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410273 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-sys-fs\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410334 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-registration-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410421 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-registration-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410425 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/68a34d3e-b358-4ff4-88a5-bd3f69f33e55-konnectivity-ca\") pod \"konnectivity-agent-cphc7\" (UID: \"68a34d3e-b358-4ff4-88a5-bd3f69f33e55\") " pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410466 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410490 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cnibin\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410505 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-systemd\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410535 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-host\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410636 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-device-dir\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410649 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysctl-d\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410706 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-run\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410720 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-os-release\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410757 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysconfig\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410740 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-var-lib-kubelet\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410784 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-sys-fs\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410808 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91084e9b-41a4-406a-8e99-40b269a9082f-tmp\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410827 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db21c734-4ec2-4cf8-9d3b-5a3095303e39-host-slash\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410841 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-var-lib-kubelet\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410846 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-lib-modules\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410892 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db21c734-4ec2-4cf8-9d3b-5a3095303e39-host-slash\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410902 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-modprobe-d\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410927 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2dx2\" (UniqueName: \"kubernetes.io/projected/91084e9b-41a4-406a-8e99-40b269a9082f-kube-api-access-b2dx2\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410957 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.410973 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410988 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysctl-conf\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.410995 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.411003 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-etc-selinux\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.411015 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cni-binary-copy\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.411940 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.411032 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs podName:b4490358-1756-4a8e-b8a6-d2503b9fdf31 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:23.911011616 +0000 UTC m=+3.137694530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs") pod "network-metrics-daemon-zrcm7" (UID: "b4490358-1756-4a8e-b8a6-d2503b9fdf31") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:23.412805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.411126 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-modprobe-d\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.412805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.411139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db21c734-4ec2-4cf8-9d3b-5a3095303e39-iptables-alerter-script\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.412805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.411153 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91084e9b-41a4-406a-8e99-40b269a9082f-etc-sysctl-conf\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.412805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.411273 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.412805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.411681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db21c734-4ec2-4cf8-9d3b-5a3095303e39-iptables-alerter-script\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.412805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.411778 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-cni-binary-copy\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.412805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.412607 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91084e9b-41a4-406a-8e99-40b269a9082f-etc-tuned\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.413563 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.413541 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/68a34d3e-b358-4ff4-88a5-bd3f69f33e55-agent-certs\") pod \"konnectivity-agent-cphc7\" (UID: \"68a34d3e-b358-4ff4-88a5-bd3f69f33e55\") " pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:23.413955 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.413924 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91084e9b-41a4-406a-8e99-40b269a9082f-tmp\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.418778 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.418495 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:54:23.418778 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.418520 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:54:23.418778 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.418534 2564 projected.go:196] Error preparing data for projected volume kube-api-access-bq6sl for pod openshift-network-diagnostics/network-check-target-gz22v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:23.418778 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.418605 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl podName:a5fa31a2-e916-4f6b-b0d7-4653f7f89d16 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:23.918582023 +0000 UTC m=+3.145264928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bq6sl" (UniqueName: "kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl") pod "network-check-target-gz22v" (UID: "a5fa31a2-e916-4f6b-b0d7-4653f7f89d16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:23.421666 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.421559 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2dx2\" (UniqueName: \"kubernetes.io/projected/91084e9b-41a4-406a-8e99-40b269a9082f-kube-api-access-b2dx2\") pod \"tuned-ftx77\" (UID: \"91084e9b-41a4-406a-8e99-40b269a9082f\") " pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.421666 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.421574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwfpr\" (UniqueName: \"kubernetes.io/projected/b4490358-1756-4a8e-b8a6-d2503b9fdf31-kube-api-access-nwfpr\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:23.421854 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.421794 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh8mk\" (UniqueName: \"kubernetes.io/projected/db21c734-4ec2-4cf8-9d3b-5a3095303e39-kube-api-access-wh8mk\") pod \"iptables-alerter-89cc4\" (UID: \"db21c734-4ec2-4cf8-9d3b-5a3095303e39\") " pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.422586 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.422513 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv7q8\" (UniqueName: \"kubernetes.io/projected/28a8a35c-90eb-45df-a836-20dd1fdd9ce8-kube-api-access-dv7q8\") pod \"multus-additional-cni-plugins-7nrrq\" (UID: \"28a8a35c-90eb-45df-a836-20dd1fdd9ce8\") " pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.423413 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.423345 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cfb7\" (UniqueName: \"kubernetes.io/projected/b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c-kube-api-access-6cfb7\") pod \"aws-ebs-csi-driver-node-ncbf8\" (UID: \"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.505737 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.505699 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9nmxt" Jul 14 13:54:23.512672 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.512634 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:23.527997 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.527968 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pf526" Jul 14 13:54:23.536463 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.536436 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lp6xp" Jul 14 13:54:23.543839 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.543817 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" Jul 14 13:54:23.551383 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.551364 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ftx77" Jul 14 13:54:23.559835 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.559813 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" Jul 14 13:54:23.567305 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.567286 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-89cc4" Jul 14 13:54:23.572154 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.572133 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 13:54:23.572243 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.572186 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:23.814277 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.814184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:23.814444 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.814366 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:23.814517 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.814450 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret podName:e4f6a09e-45c4-4875-908a-18f27e9c527c nodeName:}" failed. No retries permitted until 2026-07-14 13:54:24.814428805 +0000 UTC m=+4.041111706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret") pod "global-pull-secret-syncer-w8zzc" (UID: "e4f6a09e-45c4-4875-908a-18f27e9c527c") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:23.915594 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:23.915551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:23.915784 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.915701 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:23.915784 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:23.915779 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs podName:b4490358-1756-4a8e-b8a6-d2503b9fdf31 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:24.915757476 +0000 UTC m=+4.142440377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs") pod "network-metrics-daemon-zrcm7" (UID: "b4490358-1756-4a8e-b8a6-d2503b9fdf31") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:24.016164 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:24.016114 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:24.016390 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.016336 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:54:24.016390 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.016367 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:54:24.016390 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.016383 2564 projected.go:196] Error preparing data for projected volume kube-api-access-bq6sl for pod openshift-network-diagnostics/network-check-target-gz22v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:24.016561 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.016455 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl podName:a5fa31a2-e916-4f6b-b0d7-4653f7f89d16 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:25.016435388 +0000 UTC m=+4.243118288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq6sl" (UniqueName: "kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl") pod "network-check-target-gz22v" (UID: "a5fa31a2-e916-4f6b-b0d7-4653f7f89d16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:24.240688 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:24.240574 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 13:49:23 +0000 UTC" deadline="2028-04-24 05:16:17.767595248 +0000 UTC" Jul 14 13:54:24.240688 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:24.240633 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15591h21m53.526964841s" Jul 14 13:54:24.364844 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:24.364807 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:24.364844 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:24.364851 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:24.365032 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.364932 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:24.365076 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.365055 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:24.528553 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.528523 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a34d3e_b358_4ff4_88a5_bd3f69f33e55.slice/crio-52a56ef9690d365821629903b55842b225d454c2157a3137c61b854b75eabd82 WatchSource:0}: Error finding container 52a56ef9690d365821629903b55842b225d454c2157a3137c61b854b75eabd82: Status 404 returned error can't find the container with id 52a56ef9690d365821629903b55842b225d454c2157a3137c61b854b75eabd82 Jul 14 13:54:24.534691 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.534663 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ce727b_2258_4fd2_a90e_5ae0bb1bc22c.slice/crio-57d566e688c15fc5570b9fb7cf2de321f461ac453d82c8bf98610e671f40011b WatchSource:0}: Error finding container 57d566e688c15fc5570b9fb7cf2de321f461ac453d82c8bf98610e671f40011b: Status 404 returned error can't find the container with id 57d566e688c15fc5570b9fb7cf2de321f461ac453d82c8bf98610e671f40011b Jul 14 13:54:24.535419 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.535387 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91084e9b_41a4_406a_8e99_40b269a9082f.slice/crio-1aa088850a18c847baa4023f76b65e9ef515e6370533af94f67f9b9d1c4622bb WatchSource:0}: Error finding container 1aa088850a18c847baa4023f76b65e9ef515e6370533af94f67f9b9d1c4622bb: Status 404 returned error can't find the container with id 1aa088850a18c847baa4023f76b65e9ef515e6370533af94f67f9b9d1c4622bb Jul 14 13:54:24.537744 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.537714 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb21c734_4ec2_4cf8_9d3b_5a3095303e39.slice/crio-5b88ed892079e66555fd4a2e729708a1f70f0b5c2a97a6ffc018d4e3c3641c84 WatchSource:0}: Error finding container 5b88ed892079e66555fd4a2e729708a1f70f0b5c2a97a6ffc018d4e3c3641c84: Status 404 returned error can't find the container with id 5b88ed892079e66555fd4a2e729708a1f70f0b5c2a97a6ffc018d4e3c3641c84 Jul 14 13:54:24.538043 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.538019 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6954d52c_d756_4559_8c23_5a25abc731f0.slice/crio-47cce46c5c391cc175517aed389be9cf1874868620eb4df6ac13d86a7e30ea34 WatchSource:0}: Error finding container 47cce46c5c391cc175517aed389be9cf1874868620eb4df6ac13d86a7e30ea34: Status 404 returned error can't find the container with id 47cce46c5c391cc175517aed389be9cf1874868620eb4df6ac13d86a7e30ea34 Jul 14 13:54:24.539360 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.539323 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfe91ad7_93e0_48de_b5b5_58915fce187e.slice/crio-22ec366bf36dcbae8a240b1aa9fb8a7e095d8efe714996e08e19de2926977ac3 WatchSource:0}: Error finding container 22ec366bf36dcbae8a240b1aa9fb8a7e095d8efe714996e08e19de2926977ac3: Status 404 returned error can't find the container with id 22ec366bf36dcbae8a240b1aa9fb8a7e095d8efe714996e08e19de2926977ac3 Jul 14 13:54:24.541033 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.540963 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a8a35c_90eb_45df_a836_20dd1fdd9ce8.slice/crio-27f121fb41cbd21df7c5c488a59739c4ce6692995f0a7893d4dc6983f4051118 WatchSource:0}: Error finding container 27f121fb41cbd21df7c5c488a59739c4ce6692995f0a7893d4dc6983f4051118: Status 404 returned error can't find the container with id 27f121fb41cbd21df7c5c488a59739c4ce6692995f0a7893d4dc6983f4051118 Jul 14 13:54:24.542178 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.542116 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9817318_85dd_4754_bc4e_c2222ae011f6.slice/crio-56a879f829680ae8f7865c66ffc4b61aa93704e196f81455c28258161489775f WatchSource:0}: Error finding container 56a879f829680ae8f7865c66ffc4b61aa93704e196f81455c28258161489775f: Status 404 returned error can't find the container with id 56a879f829680ae8f7865c66ffc4b61aa93704e196f81455c28258161489775f Jul 14 13:54:24.543855 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:54:24.543826 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod305e2065_0eb3_424c_b31f_50944b568790.slice/crio-571e3c6ea23ed5160269f9d6d8e08ae7e547d36eafd5b15ad7b7e7cc41261d9a WatchSource:0}: Error finding container 571e3c6ea23ed5160269f9d6d8e08ae7e547d36eafd5b15ad7b7e7cc41261d9a: Status 404 returned error can't find the container with id 571e3c6ea23ed5160269f9d6d8e08ae7e547d36eafd5b15ad7b7e7cc41261d9a Jul 14 13:54:24.822184 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:24.821978 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:24.822329 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.822142 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:24.822329 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.822253 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret podName:e4f6a09e-45c4-4875-908a-18f27e9c527c nodeName:}" failed. No retries permitted until 2026-07-14 13:54:26.822237027 +0000 UTC m=+6.048919927 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret") pod "global-pull-secret-syncer-w8zzc" (UID: "e4f6a09e-45c4-4875-908a-18f27e9c527c") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:24.923436 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:24.923403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:24.923594 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.923512 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:24.923594 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:24.923573 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs podName:b4490358-1756-4a8e-b8a6-d2503b9fdf31 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:26.923552638 +0000 UTC m=+6.150235541 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs") pod "network-metrics-daemon-zrcm7" (UID: "b4490358-1756-4a8e-b8a6-d2503b9fdf31") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:25.024908 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.024176 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:25.024908 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:25.024420 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:54:25.024908 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:25.024443 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:54:25.024908 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:25.024457 2564 projected.go:196] Error preparing data for projected volume kube-api-access-bq6sl for pod openshift-network-diagnostics/network-check-target-gz22v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:25.024908 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:25.024519 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl podName:a5fa31a2-e916-4f6b-b0d7-4653f7f89d16 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:27.024499498 +0000 UTC m=+6.251182399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq6sl" (UniqueName: "kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl") pod "network-check-target-gz22v" (UID: "a5fa31a2-e916-4f6b-b0d7-4653f7f89d16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:25.240814 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.240766 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 13:49:23 +0000 UTC" deadline="2028-02-28 21:10:46.46868101 +0000 UTC" Jul 14 13:54:25.240814 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.240812 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14263h16m21.227873658s" Jul 14 13:54:25.364611 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.364528 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:25.364769 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:25.364697 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:25.404918 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.404868 2564 generic.go:356] "Generic (PLEG): container finished" podID="26313c9496ac2ecad22f089933c11a34" containerID="b640fcd99c0a1beaa83c2c81a3475910335796cc19506471192c974cfde0fd32" exitCode=0 Jul 14 13:54:25.405087 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.404974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" event={"ID":"26313c9496ac2ecad22f089933c11a34","Type":"ContainerDied","Data":"b640fcd99c0a1beaa83c2c81a3475910335796cc19506471192c974cfde0fd32"} Jul 14 13:54:25.418873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.418832 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" event={"ID":"e38e3cc78bf216faf085a021c57364a2","Type":"ContainerStarted","Data":"8acc98b5c72c364269a4f44257e482c5200a5386506bef3a13e99bb17976094e"} Jul 14 13:54:25.434135 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.432755 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9nmxt" event={"ID":"305e2065-0eb3-424c-b31f-50944b568790","Type":"ContainerStarted","Data":"571e3c6ea23ed5160269f9d6d8e08ae7e547d36eafd5b15ad7b7e7cc41261d9a"} Jul 14 13:54:25.451600 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.451522 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pf526" event={"ID":"f9817318-85dd-4754-bc4e-c2222ae011f6","Type":"ContainerStarted","Data":"56a879f829680ae8f7865c66ffc4b61aa93704e196f81455c28258161489775f"} Jul 14 13:54:25.468833 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.468731 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerStarted","Data":"27f121fb41cbd21df7c5c488a59739c4ce6692995f0a7893d4dc6983f4051118"} Jul 14 13:54:25.491811 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.491688 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"47cce46c5c391cc175517aed389be9cf1874868620eb4df6ac13d86a7e30ea34"} Jul 14 13:54:25.506311 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.506009 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lp6xp" event={"ID":"dfe91ad7-93e0-48de-b5b5-58915fce187e","Type":"ContainerStarted","Data":"22ec366bf36dcbae8a240b1aa9fb8a7e095d8efe714996e08e19de2926977ac3"} Jul 14 13:54:25.518563 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.518469 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-89cc4" event={"ID":"db21c734-4ec2-4cf8-9d3b-5a3095303e39","Type":"ContainerStarted","Data":"5b88ed892079e66555fd4a2e729708a1f70f0b5c2a97a6ffc018d4e3c3641c84"} Jul 14 13:54:25.521051 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.520985 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ftx77" event={"ID":"91084e9b-41a4-406a-8e99-40b269a9082f","Type":"ContainerStarted","Data":"1aa088850a18c847baa4023f76b65e9ef515e6370533af94f67f9b9d1c4622bb"} Jul 14 13:54:25.545322 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.545215 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" event={"ID":"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c","Type":"ContainerStarted","Data":"57d566e688c15fc5570b9fb7cf2de321f461ac453d82c8bf98610e671f40011b"} Jul 14 13:54:25.566316 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:25.566223 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cphc7" event={"ID":"68a34d3e-b358-4ff4-88a5-bd3f69f33e55","Type":"ContainerStarted","Data":"52a56ef9690d365821629903b55842b225d454c2157a3137c61b854b75eabd82"} Jul 14 13:54:26.364059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:26.364028 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:26.364616 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:26.364157 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:26.364616 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:26.364578 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:26.364752 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:26.364677 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:26.598078 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:26.598037 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" event={"ID":"26313c9496ac2ecad22f089933c11a34","Type":"ContainerStarted","Data":"889790b8a0737544f8617519b194a72c0eee846be2bba45872d67a84f02a2ecb"} Jul 14 13:54:26.616553 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:26.616442 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-141.ec2.internal" podStartSLOduration=4.616417416 podStartE2EDuration="4.616417416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:25.443722937 +0000 UTC m=+4.670405862" watchObservedRunningTime="2026-07-14 13:54:26.616417416 +0000 UTC m=+5.843100339" Jul 14 13:54:26.844733 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:26.844690 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:26.844917 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:26.844860 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:26.844982 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:26.844923 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret podName:e4f6a09e-45c4-4875-908a-18f27e9c527c nodeName:}" failed. No retries permitted until 2026-07-14 13:54:30.844903628 +0000 UTC m=+10.071586537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret") pod "global-pull-secret-syncer-w8zzc" (UID: "e4f6a09e-45c4-4875-908a-18f27e9c527c") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:26.945229 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:26.945114 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:26.945863 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:26.945321 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:26.945863 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:26.945485 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs podName:b4490358-1756-4a8e-b8a6-d2503b9fdf31 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:30.94546419 +0000 UTC m=+10.172147087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs") pod "network-metrics-daemon-zrcm7" (UID: "b4490358-1756-4a8e-b8a6-d2503b9fdf31") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:27.046520 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:27.045790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:27.046520 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:27.046012 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:54:27.046520 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:27.046035 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:54:27.046520 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:27.046047 2564 projected.go:196] Error preparing data for projected volume kube-api-access-bq6sl for pod openshift-network-diagnostics/network-check-target-gz22v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:27.046520 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:27.046105 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl podName:a5fa31a2-e916-4f6b-b0d7-4653f7f89d16 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:31.046085239 +0000 UTC m=+10.272768146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq6sl" (UniqueName: "kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl") pod "network-check-target-gz22v" (UID: "a5fa31a2-e916-4f6b-b0d7-4653f7f89d16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:27.365454 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:27.364877 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:27.365454 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:27.365041 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:28.363849 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:28.363811 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:28.364027 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:28.363946 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:28.364027 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:28.363993 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:28.364161 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:28.364108 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:29.365629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:29.365221 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:29.365629 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:29.365389 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:30.364622 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:30.364542 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:30.364814 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:30.364684 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:30.365099 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:30.364931 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:30.365099 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:30.365062 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:30.880825 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:30.880297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:30.880825 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:30.880432 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:30.880825 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:30.880494 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret podName:e4f6a09e-45c4-4875-908a-18f27e9c527c nodeName:}" failed. No retries permitted until 2026-07-14 13:54:38.880476472 +0000 UTC m=+18.107159367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret") pod "global-pull-secret-syncer-w8zzc" (UID: "e4f6a09e-45c4-4875-908a-18f27e9c527c") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:30.981642 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:30.981550 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:30.981783 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:30.981697 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:30.981856 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:30.981782 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs podName:b4490358-1756-4a8e-b8a6-d2503b9fdf31 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:38.981760336 +0000 UTC m=+18.208443242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs") pod "network-metrics-daemon-zrcm7" (UID: "b4490358-1756-4a8e-b8a6-d2503b9fdf31") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:31.083181 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:31.082518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:31.083181 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:31.082751 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:54:31.083181 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:31.082775 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:54:31.083181 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:31.082788 2564 projected.go:196] Error preparing data for projected volume kube-api-access-bq6sl for pod openshift-network-diagnostics/network-check-target-gz22v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:31.083181 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:31.082847 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl podName:a5fa31a2-e916-4f6b-b0d7-4653f7f89d16 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:39.082828921 +0000 UTC m=+18.309511830 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq6sl" (UniqueName: "kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl") pod "network-check-target-gz22v" (UID: "a5fa31a2-e916-4f6b-b0d7-4653f7f89d16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:31.366049 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:31.365536 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:31.366049 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:31.365661 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:32.364019 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:32.363906 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:32.364535 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:32.364091 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:32.364535 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:32.364196 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:32.364535 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:32.364332 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:33.364971 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:33.364740 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:33.364971 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:33.364885 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:34.364600 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:34.364507 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:34.364787 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:34.364651 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:34.364787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:34.364507 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:34.364787 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:34.364767 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:35.364202 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:35.364120 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:35.364671 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:35.364265 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:36.364505 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:36.364463 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:36.364908 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:36.364577 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:36.364908 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:36.364629 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:36.364908 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:36.364742 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:37.364714 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:37.364680 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:37.365128 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:37.364811 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:38.364427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:38.364387 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:38.364694 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:38.364512 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:38.364694 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:38.364564 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:38.364694 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:38.364680 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:38.946353 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:38.946316 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:38.946518 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:38.946452 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:38.946573 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:38.946524 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret podName:e4f6a09e-45c4-4875-908a-18f27e9c527c nodeName:}" failed. No retries permitted until 2026-07-14 13:54:54.946503209 +0000 UTC m=+34.173186121 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret") pod "global-pull-secret-syncer-w8zzc" (UID: "e4f6a09e-45c4-4875-908a-18f27e9c527c") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:39.047425 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:39.047393 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:39.047578 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:39.047550 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:39.047650 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:39.047611 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs podName:b4490358-1756-4a8e-b8a6-d2503b9fdf31 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:55.047594301 +0000 UTC m=+34.274277212 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs") pod "network-metrics-daemon-zrcm7" (UID: "b4490358-1756-4a8e-b8a6-d2503b9fdf31") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:39.147816 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:39.147721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:39.147981 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:39.147907 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:54:39.147981 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:39.147931 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:54:39.147981 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:39.147944 2564 projected.go:196] Error preparing data for projected volume kube-api-access-bq6sl for pod openshift-network-diagnostics/network-check-target-gz22v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:39.148191 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:39.147999 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl podName:a5fa31a2-e916-4f6b-b0d7-4653f7f89d16 nodeName:}" failed. No retries permitted until 2026-07-14 13:54:55.147980983 +0000 UTC m=+34.374663879 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq6sl" (UniqueName: "kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl") pod "network-check-target-gz22v" (UID: "a5fa31a2-e916-4f6b-b0d7-4653f7f89d16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:39.364741 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:39.364707 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:39.364903 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:39.364849 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:40.364466 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:40.364427 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:40.364640 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:40.364422 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:40.364640 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:40.364540 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:40.364718 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:40.364654 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:41.365144 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:41.365107 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:41.365570 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:41.365242 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:42.364625 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:42.364586 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:42.364625 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:42.364625 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:42.364830 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:42.364702 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:42.364885 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:42.364840 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:43.364788 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.364760 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:43.365103 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:43.364885 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:43.631495 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.631333 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9nmxt" event={"ID":"305e2065-0eb3-424c-b31f-50944b568790","Type":"ContainerStarted","Data":"b558760b1a0a47a5cd2a2642fb6de40cd404bb0cc80fb8a538e5bc9dc158e4e3"} Jul 14 13:54:43.636545 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.636513 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pf526" event={"ID":"f9817318-85dd-4754-bc4e-c2222ae011f6","Type":"ContainerStarted","Data":"b1034a4e4d32dc5d5fbaf48d1bd1c54fc3856086f5b83aef4b11ead11f8024c5"} Jul 14 13:54:43.637983 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.637956 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerStarted","Data":"c413164dc16e867166dfe0a7f8f34ba327bfb4da7a27f327af49dbe4d11a6932"} Jul 14 13:54:43.639331 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.639307 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"4e226a27a06ca3902a5cc058a623c9dada2c6a8a45672509daaa0ab46ff61f25"} Jul 14 13:54:43.640499 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.640472 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lp6xp" event={"ID":"dfe91ad7-93e0-48de-b5b5-58915fce187e","Type":"ContainerStarted","Data":"7a1629fae6f6968a4b10f619b8b8edba538ff67c558f2e6aa811259f8a902e72"} Jul 14 13:54:43.641808 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.641790 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ftx77" event={"ID":"91084e9b-41a4-406a-8e99-40b269a9082f","Type":"ContainerStarted","Data":"bdc5bbd5c054e9b50c272d2a200008e4839d77d6352fa46fc6d6838c857aa6e9"} Jul 14 13:54:43.643249 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.643227 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" event={"ID":"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c","Type":"ContainerStarted","Data":"0b5479c60c7e7a29b5883a026133573ce9b45d3c292f8930d449882cad36bd1b"} Jul 14 13:54:43.645360 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.645339 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cphc7" event={"ID":"68a34d3e-b358-4ff4-88a5-bd3f69f33e55","Type":"ContainerStarted","Data":"3353f04c6bf1c21f1df6507f0ff681c7d5ce8548b760eb4057e27a8627509daa"} Jul 14 13:54:43.649814 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.649722 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-141.ec2.internal" podStartSLOduration=21.649711943 podStartE2EDuration="21.649711943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:26.617447727 +0000 UTC m=+5.844130680" watchObservedRunningTime="2026-07-14 13:54:43.649711943 +0000 UTC m=+22.876394860" Jul 14 13:54:43.650009 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.649986 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9nmxt" podStartSLOduration=3.8767427740000002 podStartE2EDuration="22.649981034s" totalImagesPullingTime="18.77323826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:43.649660434 +0000 UTC m=+22.876343352" watchObservedRunningTime="2026-07-14 13:54:43.649981034 +0000 UTC m=+22.876663950" Jul 14 13:54:43.670336 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.670297 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pf526" podStartSLOduration=3.832106406 podStartE2EDuration="22.670266889s" totalImagesPullingTime="18.838160483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:43.670165545 +0000 UTC m=+22.896848481" watchObservedRunningTime="2026-07-14 13:54:43.670266889 +0000 UTC m=+22.896949848" Jul 14 13:54:43.692141 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.692098 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ftx77" podStartSLOduration=3.8262661700000002 podStartE2EDuration="22.692083216s" totalImagesPullingTime="18.865817046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:43.691576819 +0000 UTC m=+22.918259736" watchObservedRunningTime="2026-07-14 13:54:43.692083216 +0000 UTC m=+22.918766132" Jul 14 13:54:43.731635 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.731582 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lp6xp" podStartSLOduration=3.932541364 podStartE2EDuration="22.731558458s" totalImagesPullingTime="18.799017094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:43.731145224 +0000 UTC m=+22.957828163" watchObservedRunningTime="2026-07-14 13:54:43.731558458 +0000 UTC m=+22.958241381" Jul 14 13:54:43.749079 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:43.749038 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cphc7" podStartSLOduration=8.458198379 podStartE2EDuration="22.749021095s" totalImagesPullingTime="14.290822716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:43.748962034 +0000 UTC m=+22.975644976" watchObservedRunningTime="2026-07-14 13:54:43.749021095 +0000 UTC m=+22.975704009" Jul 14 13:54:44.364901 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.364687 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:44.365585 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.364687 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:44.365585 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:44.365009 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:44.365585 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:44.365059 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:44.650721 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.650695 2564 generic.go:356] "Generic (PLEG): container finished" podID="28a8a35c-90eb-45df-a836-20dd1fdd9ce8" containerID="c413164dc16e867166dfe0a7f8f34ba327bfb4da7a27f327af49dbe4d11a6932" exitCode=0 Jul 14 13:54:44.650843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.650777 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerDied","Data":"c413164dc16e867166dfe0a7f8f34ba327bfb4da7a27f327af49dbe4d11a6932"} Jul 14 13:54:44.653524 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.653507 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 13:54:44.653841 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.653821 2564 generic.go:356] "Generic (PLEG): container finished" podID="6954d52c-d756-4559-8c23-5a25abc731f0" containerID="e04635167f4bc334c63775af3674d801fad4a56db00b709a7a05cda6dd8a40ce" exitCode=1 Jul 14 13:54:44.653926 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.653902 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"8e8449fa757ac6478d8943d7e6bfa2d66ac103bf40d02e4347ef12c1174e72b0"} Jul 14 13:54:44.654009 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.653995 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"5b83819e424f0c736a2ca1cc016562dfe049dca6108438cab5c15055a29482fa"} Jul 14 13:54:44.654059 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.654048 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"a1751f3d21a3e2e03856a631ea14c9f3495646466ace53d388aed2d3cd5488a8"} Jul 14 13:54:44.654092 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.654083 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"b7cd371f1ecea759fa77f2ec1cc15dea33ad3d83fb68f752e155353c4c322b49"} Jul 14 13:54:44.654124 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.654112 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerDied","Data":"e04635167f4bc334c63775af3674d801fad4a56db00b709a7a05cda6dd8a40ce"} Jul 14 13:54:44.958520 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:44.958490 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 13:54:45.273090 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:45.272946 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T13:54:44.958514183Z","UUID":"559e6ee2-a1d8-441e-914b-41bf30e9522c","Handler":null,"Name":"","Endpoint":""} Jul 14 13:54:45.274869 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:45.274846 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 13:54:45.274994 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:45.274889 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 13:54:45.364897 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:45.364861 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:45.365064 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:45.365008 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:45.660531 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:45.660492 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" event={"ID":"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c","Type":"ContainerStarted","Data":"8f87388bb637dec653112eca0895ba7c777f93625369f995136551936f514d18"} Jul 14 13:54:45.662265 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:45.662224 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-89cc4" event={"ID":"db21c734-4ec2-4cf8-9d3b-5a3095303e39","Type":"ContainerStarted","Data":"77b719a4516de3b1c10a944252458fe1c62fa85efb055d1ea49086096c5d2e95"} Jul 14 13:54:45.679465 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:45.679420 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-89cc4" podStartSLOduration=5.817301578 podStartE2EDuration="24.679405648s" totalImagesPullingTime="18.86210407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:45.67890593 +0000 UTC m=+24.905588849" watchObservedRunningTime="2026-07-14 13:54:45.679405648 +0000 UTC m=+24.906088565" Jul 14 13:54:46.364414 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:46.364183 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:46.364589 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:46.364183 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:46.364589 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:46.364514 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:46.364589 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:46.364556 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:46.667807 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:46.667777 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" event={"ID":"b4ce727b-2258-4fd2-a90e-5ae0bb1bc22c","Type":"ContainerStarted","Data":"24216005da6ed0355c424d03dcc4c6a54139fa3af30dc089af4f48cadf5df5ff"} Jul 14 13:54:46.671436 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:46.671415 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 13:54:46.671835 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:46.671809 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"b9d97a8aafd3597b88756dc30a1798f62b39d8d3d69798f396e9921cd4bc3ac9"} Jul 14 13:54:47.364157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:47.364124 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:47.364408 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:47.364246 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:48.364602 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:48.364571 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:48.365166 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:48.364752 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:48.365166 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:48.364782 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:48.365166 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:48.364865 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:48.554966 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:48.554931 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:48.555595 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:48.555573 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:48.578266 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:48.578218 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ncbf8" podStartSLOduration=6.238522443 podStartE2EDuration="27.578198143s" totalImagesPullingTime="21.3396757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:46.692001903 +0000 UTC m=+25.918684829" watchObservedRunningTime="2026-07-14 13:54:48.578198143 +0000 UTC m=+27.804881083" Jul 14 13:54:48.676974 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:48.676872 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:48.677470 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:48.677437 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cphc7" Jul 14 13:54:49.364356 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:49.364157 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:49.364520 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:49.364454 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:49.680556 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:49.680524 2564 generic.go:356] "Generic (PLEG): container finished" podID="28a8a35c-90eb-45df-a836-20dd1fdd9ce8" containerID="f26f1f543e67243eeb94feceed73ac65b51d12daec79148004498e6f46e35818" exitCode=0 Jul 14 13:54:49.681314 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:49.680614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerDied","Data":"f26f1f543e67243eeb94feceed73ac65b51d12daec79148004498e6f46e35818"} Jul 14 13:54:49.684237 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:49.684217 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 13:54:49.684600 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:49.684575 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"f5dcfdff00c351cb7404f148f5b45c1ed113b4163a77747b3c0a2ec65197f0f4"} Jul 14 13:54:49.685016 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:49.685001 2564 scope.go:117] "RemoveContainer" containerID="e04635167f4bc334c63775af3674d801fad4a56db00b709a7a05cda6dd8a40ce" Jul 14 13:54:50.364442 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.364414 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:50.364633 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.364415 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:50.364633 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:50.364518 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:50.364633 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:50.364583 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:50.689873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.689797 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 13:54:50.690414 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.690192 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" event={"ID":"6954d52c-d756-4559-8c23-5a25abc731f0","Type":"ContainerStarted","Data":"2941a728791931c766becd4c8feaad8644da58b4aa9dcef11ed2ed2d0f8aff40"} Jul 14 13:54:50.690414 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.690362 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:50.690537 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.690441 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:50.690537 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.690489 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:50.705826 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.705804 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:50.705951 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.705905 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:54:50.721536 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:50.721492 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" podStartSLOduration=10.739020732 podStartE2EDuration="29.721477038s" totalImagesPullingTime="18.982456306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:54:50.72116293 +0000 UTC m=+29.947845856" watchObservedRunningTime="2026-07-14 13:54:50.721477038 +0000 UTC m=+29.948159954" Jul 14 13:54:51.365952 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:51.365918 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:51.366111 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:51.366047 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:51.693955 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:51.693866 2564 generic.go:356] "Generic (PLEG): container finished" podID="28a8a35c-90eb-45df-a836-20dd1fdd9ce8" containerID="5e290e9606a8b13b3e1c366265830eb7a667b1c30fd105d4a007195e1f1bc80b" exitCode=0 Jul 14 13:54:51.693955 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:51.693945 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerDied","Data":"5e290e9606a8b13b3e1c366265830eb7a667b1c30fd105d4a007195e1f1bc80b"} Jul 14 13:54:52.364787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:52.364754 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:52.365306 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:52.364757 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:52.365306 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:52.364883 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:52.365306 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:52.364919 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:53.364348 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:53.364309 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:53.364741 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:53.364459 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:53.701491 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:53.701409 2564 generic.go:356] "Generic (PLEG): container finished" podID="28a8a35c-90eb-45df-a836-20dd1fdd9ce8" containerID="53c68afe219cb3c9aeab235047f31c2cfb54bd6e344bbc2c0ccf42fe799ca4aa" exitCode=0 Jul 14 13:54:53.701655 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:53.701484 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerDied","Data":"53c68afe219cb3c9aeab235047f31c2cfb54bd6e344bbc2c0ccf42fe799ca4aa"} Jul 14 13:54:54.363877 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:54.363843 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:54.364072 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:54.363846 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:54.364072 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:54.363961 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:54.364186 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:54.364065 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:54.964449 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:54.964414 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:54.964850 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:54.964558 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:54.964850 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:54.964633 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret podName:e4f6a09e-45c4-4875-908a-18f27e9c527c nodeName:}" failed. No retries permitted until 2026-07-14 13:55:26.964614387 +0000 UTC m=+66.191297290 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret") pod "global-pull-secret-syncer-w8zzc" (UID: "e4f6a09e-45c4-4875-908a-18f27e9c527c") : object "kube-system"/"original-pull-secret" not registered Jul 14 13:54:55.065061 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:55.065023 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:55.065242 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:55.065205 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:55.065320 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:55.065298 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs podName:b4490358-1756-4a8e-b8a6-d2503b9fdf31 nodeName:}" failed. No retries permitted until 2026-07-14 13:55:27.06526383 +0000 UTC m=+66.291946740 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs") pod "network-metrics-daemon-zrcm7" (UID: "b4490358-1756-4a8e-b8a6-d2503b9fdf31") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 13:54:55.166403 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:55.166366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:55.166583 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:55.166520 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 13:54:55.166583 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:55.166546 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 13:54:55.166583 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:55.166556 2564 projected.go:196] Error preparing data for projected volume kube-api-access-bq6sl for pod openshift-network-diagnostics/network-check-target-gz22v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:55.166773 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:55.166612 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl podName:a5fa31a2-e916-4f6b-b0d7-4653f7f89d16 nodeName:}" failed. No retries permitted until 2026-07-14 13:55:27.166595682 +0000 UTC m=+66.393278597 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq6sl" (UniqueName: "kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl") pod "network-check-target-gz22v" (UID: "a5fa31a2-e916-4f6b-b0d7-4653f7f89d16") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 13:54:55.364177 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:55.364134 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:55.364371 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:55.364273 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:56.364996 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:56.364789 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:56.365456 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:56.364790 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:56.365456 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:56.365100 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:56.365456 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:56.365200 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:57.364736 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:57.364701 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:57.364919 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:57.364828 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:58.364646 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:58.364615 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:54:58.365027 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:58.364615 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:54:58.365027 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:58.364710 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:54:58.365027 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:58.364788 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:54:58.715404 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:58.715320 2564 generic.go:356] "Generic (PLEG): container finished" podID="28a8a35c-90eb-45df-a836-20dd1fdd9ce8" containerID="70cf5c638ee6b4d04a54e45aacba34f33a2baa9e5023602a9746415ee36951e8" exitCode=0 Jul 14 13:54:58.715404 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:58.715367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerDied","Data":"70cf5c638ee6b4d04a54e45aacba34f33a2baa9e5023602a9746415ee36951e8"} Jul 14 13:54:59.364821 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:59.364787 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:54:59.365225 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:54:59.364905 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:54:59.720226 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:59.720147 2564 generic.go:356] "Generic (PLEG): container finished" podID="28a8a35c-90eb-45df-a836-20dd1fdd9ce8" containerID="edbff5c55de508ea20efa72a2d41a6bb653c33afdb8492cf1601068bf7282841" exitCode=0 Jul 14 13:54:59.720226 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:54:59.720197 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerDied","Data":"edbff5c55de508ea20efa72a2d41a6bb653c33afdb8492cf1601068bf7282841"} Jul 14 13:55:00.364333 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:00.364295 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:55:00.364480 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:00.364295 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:00.364480 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:00.364401 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:55:00.364567 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:00.364511 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:55:00.725567 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:00.725490 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" event={"ID":"28a8a35c-90eb-45df-a836-20dd1fdd9ce8","Type":"ContainerStarted","Data":"21240ff8af5b60abac913c5ba102fa0f0395c55e4b590377ea216d25e020fcaa"} Jul 14 13:55:00.752794 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:00.752745 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7nrrq" podStartSLOduration=9.931975835 podStartE2EDuration="39.75273083s" totalImagesPullingTime="29.820754995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:00.750908354 +0000 UTC m=+39.977591285" watchObservedRunningTime="2026-07-14 13:55:00.75273083 +0000 UTC m=+39.979413746" Jul 14 13:55:01.365406 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:01.365374 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:55:01.365602 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:01.365468 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:55:02.364817 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:02.364782 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:02.365187 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:02.364887 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:55:02.365187 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:02.364952 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:55:02.365187 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:02.365035 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:55:03.238079 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:03.238047 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zrcm7"] Jul 14 13:55:03.238301 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:03.238219 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:55:03.238398 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:03.238371 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:55:03.238879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:03.238844 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w8zzc"] Jul 14 13:55:03.239004 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:03.238949 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:55:03.239068 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:03.239039 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:55:03.239376 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:03.239352 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gz22v"] Jul 14 13:55:03.239472 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:03.239433 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:03.239514 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:03.239494 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:55:05.364474 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:05.364296 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:55:05.365046 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:05.364303 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:05.365046 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:05.364556 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w8zzc" podUID="e4f6a09e-45c4-4875-908a-18f27e9c527c" Jul 14 13:55:05.365046 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:05.364630 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gz22v" podUID="a5fa31a2-e916-4f6b-b0d7-4653f7f89d16" Jul 14 13:55:05.365046 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:05.364307 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:55:05.365046 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:05.364714 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zrcm7" podUID="b4490358-1756-4a8e-b8a6-d2503b9fdf31" Jul 14 13:55:07.070973 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.070946 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-141.ec2.internal" event="NodeReady" Jul 14 13:55:07.071395 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.071078 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 13:55:07.128432 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.128403 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-22d2x"] Jul 14 13:55:07.157597 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.157490 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22d2x"] Jul 14 13:55:07.157597 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.157585 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22d2x" Jul 14 13:55:07.161175 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.161155 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 13:55:07.161507 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.161486 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 13:55:07.161626 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.161525 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 13:55:07.161699 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.161664 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rvh4f\"" Jul 14 13:55:07.166418 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.166395 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mdtx6"] Jul 14 13:55:07.187708 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.187681 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mdtx6"] Jul 14 13:55:07.187846 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.187802 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.191451 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.191428 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 13:55:07.191731 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.191713 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 13:55:07.191806 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.191716 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rr67m\"" Jul 14 13:55:07.191877 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.191786 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 13:55:07.191972 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.191958 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 13:55:07.211766 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.211741 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-85kbp"] Jul 14 13:55:07.231709 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.231639 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.234838 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.234809 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 13:55:07.235057 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.235040 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 13:55:07.235118 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.235099 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mh5f9\"" Jul 14 13:55:07.235118 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.235102 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-85kbp"] Jul 14 13:55:07.352537 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352493 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hpgw\" (UniqueName: \"kubernetes.io/projected/2b621f22-17e4-4eba-90e6-b84f76091ee9-kube-api-access-9hpgw\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.352704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/983e0528-5495-413e-9bcc-73e699b8e85c-cert\") pod \"ingress-canary-22d2x\" (UID: \"983e0528-5495-413e-9bcc-73e699b8e85c\") " pod="openshift-ingress-canary/ingress-canary-22d2x" Jul 14 13:55:07.352704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352602 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-tmp-dir\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.352704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352616 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b621f22-17e4-4eba-90e6-b84f76091ee9-crio-socket\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.352704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352680 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsccq\" (UniqueName: \"kubernetes.io/projected/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-kube-api-access-bsccq\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.352841 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b621f22-17e4-4eba-90e6-b84f76091ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.352841 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352733 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44756\" (UniqueName: \"kubernetes.io/projected/983e0528-5495-413e-9bcc-73e699b8e85c-kube-api-access-44756\") pod \"ingress-canary-22d2x\" (UID: \"983e0528-5495-413e-9bcc-73e699b8e85c\") " pod="openshift-ingress-canary/ingress-canary-22d2x" Jul 14 13:55:07.352841 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-metrics-tls\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.352841 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352790 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b621f22-17e4-4eba-90e6-b84f76091ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.352957 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352843 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-config-volume\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.352957 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.352866 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b621f22-17e4-4eba-90e6-b84f76091ee9-data-volume\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.364728 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.364706 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:55:07.364803 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.364789 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:07.364912 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.364896 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:55:07.368400 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.368381 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 13:55:07.368958 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.368937 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7c4p6\"" Jul 14 13:55:07.369054 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.368992 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 13:55:07.369118 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.369103 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 13:55:07.369352 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.369339 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v5knd\"" Jul 14 13:55:07.370197 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.370177 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 13:55:07.453519 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hpgw\" (UniqueName: \"kubernetes.io/projected/2b621f22-17e4-4eba-90e6-b84f76091ee9-kube-api-access-9hpgw\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.453519 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453475 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/983e0528-5495-413e-9bcc-73e699b8e85c-cert\") pod \"ingress-canary-22d2x\" (UID: \"983e0528-5495-413e-9bcc-73e699b8e85c\") " pod="openshift-ingress-canary/ingress-canary-22d2x" Jul 14 13:55:07.453519 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453492 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-tmp-dir\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.453519 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453509 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b621f22-17e4-4eba-90e6-b84f76091ee9-crio-socket\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.453873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453538 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsccq\" (UniqueName: \"kubernetes.io/projected/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-kube-api-access-bsccq\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.453873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453554 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b621f22-17e4-4eba-90e6-b84f76091ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.453873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453678 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44756\" (UniqueName: \"kubernetes.io/projected/983e0528-5495-413e-9bcc-73e699b8e85c-kube-api-access-44756\") pod \"ingress-canary-22d2x\" (UID: \"983e0528-5495-413e-9bcc-73e699b8e85c\") " pod="openshift-ingress-canary/ingress-canary-22d2x" Jul 14 13:55:07.453873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453709 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-metrics-tls\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.453873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453727 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b621f22-17e4-4eba-90e6-b84f76091ee9-crio-socket\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.453873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b621f22-17e4-4eba-90e6-b84f76091ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.453873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453825 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-config-volume\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.453873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b621f22-17e4-4eba-90e6-b84f76091ee9-data-volume\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.454260 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.453911 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-tmp-dir\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.454260 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.454165 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b621f22-17e4-4eba-90e6-b84f76091ee9-data-volume\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.454260 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.454228 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b621f22-17e4-4eba-90e6-b84f76091ee9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.454433 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.454341 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-config-volume\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.457583 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.457561 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b621f22-17e4-4eba-90e6-b84f76091ee9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.457583 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.457567 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-metrics-tls\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.457743 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.457676 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/983e0528-5495-413e-9bcc-73e699b8e85c-cert\") pod \"ingress-canary-22d2x\" (UID: \"983e0528-5495-413e-9bcc-73e699b8e85c\") " pod="openshift-ingress-canary/ingress-canary-22d2x" Jul 14 13:55:07.470266 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.470237 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hpgw\" (UniqueName: \"kubernetes.io/projected/2b621f22-17e4-4eba-90e6-b84f76091ee9-kube-api-access-9hpgw\") pod \"insights-runtime-extractor-mdtx6\" (UID: \"2b621f22-17e4-4eba-90e6-b84f76091ee9\") " pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.470377 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.470312 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsccq\" (UniqueName: \"kubernetes.io/projected/487d9f7a-54b6-4d0b-8b43-da0472dc2a3a-kube-api-access-bsccq\") pod \"dns-default-85kbp\" (UID: \"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a\") " pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.471177 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.471160 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44756\" (UniqueName: \"kubernetes.io/projected/983e0528-5495-413e-9bcc-73e699b8e85c-kube-api-access-44756\") pod \"ingress-canary-22d2x\" (UID: \"983e0528-5495-413e-9bcc-73e699b8e85c\") " pod="openshift-ingress-canary/ingress-canary-22d2x" Jul 14 13:55:07.496039 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.496008 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mdtx6" Jul 14 13:55:07.541057 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.541034 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:07.713857 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.713787 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-85kbp"] Jul 14 13:55:07.717219 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.717193 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mdtx6"] Jul 14 13:55:07.717768 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:07.717739 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod487d9f7a_54b6_4d0b_8b43_da0472dc2a3a.slice/crio-928896ee2fea27fafcfe95a88c3246dad86344061f56c87372fc2c502492969a WatchSource:0}: Error finding container 928896ee2fea27fafcfe95a88c3246dad86344061f56c87372fc2c502492969a: Status 404 returned error can't find the container with id 928896ee2fea27fafcfe95a88c3246dad86344061f56c87372fc2c502492969a Jul 14 13:55:07.719686 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:07.719666 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b621f22_17e4_4eba_90e6_b84f76091ee9.slice/crio-1f58f3129aa8e7173383731fc74f135e2abccab3859ef2ada39cea16501362b3 WatchSource:0}: Error finding container 1f58f3129aa8e7173383731fc74f135e2abccab3859ef2ada39cea16501362b3: Status 404 returned error can't find the container with id 1f58f3129aa8e7173383731fc74f135e2abccab3859ef2ada39cea16501362b3 Jul 14 13:55:07.743889 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.743863 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85kbp" event={"ID":"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a","Type":"ContainerStarted","Data":"928896ee2fea27fafcfe95a88c3246dad86344061f56c87372fc2c502492969a"} Jul 14 13:55:07.744901 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.744880 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdtx6" event={"ID":"2b621f22-17e4-4eba-90e6-b84f76091ee9","Type":"ContainerStarted","Data":"1f58f3129aa8e7173383731fc74f135e2abccab3859ef2ada39cea16501362b3"} Jul 14 13:55:07.767347 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.767321 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22d2x" Jul 14 13:55:07.914261 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:07.914234 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22d2x"] Jul 14 13:55:07.917445 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:07.917417 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod983e0528_5495_413e_9bcc_73e699b8e85c.slice/crio-a2839494d3f5733a73a043ac7a8569d2ca0c7e2636c0ac2e424fae1113ba34ab WatchSource:0}: Error finding container a2839494d3f5733a73a043ac7a8569d2ca0c7e2636c0ac2e424fae1113ba34ab: Status 404 returned error can't find the container with id a2839494d3f5733a73a043ac7a8569d2ca0c7e2636c0ac2e424fae1113ba34ab Jul 14 13:55:08.751003 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:08.750969 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22d2x" event={"ID":"983e0528-5495-413e-9bcc-73e699b8e85c","Type":"ContainerStarted","Data":"a2839494d3f5733a73a043ac7a8569d2ca0c7e2636c0ac2e424fae1113ba34ab"} Jul 14 13:55:10.043192 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.043157 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-565f9cc7f5-sns9v"] Jul 14 13:55:10.057628 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.057599 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.059547 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.059522 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-565f9cc7f5-sns9v"] Jul 14 13:55:10.061253 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.061195 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 13:55:10.061816 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.061790 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 13:55:10.062539 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.062520 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 13:55:10.062647 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.062545 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qrpw7\"" Jul 14 13:55:10.062647 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.062586 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 13:55:10.063179 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.063158 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 13:55:10.063294 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.063216 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 13:55:10.063294 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.063216 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 13:55:10.069049 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.069027 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 13:55:10.074815 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.074782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-oauth-config\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.074930 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.074837 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-service-ca\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.074989 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.074925 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj7vb\" (UniqueName: \"kubernetes.io/projected/6d9e7588-5f55-49fe-9ac6-e59d234c2235-kube-api-access-mj7vb\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.074989 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.074974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-trusted-ca-bundle\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.075095 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.075002 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-oauth-serving-cert\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.075095 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.075039 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-config\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.075188 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.075098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-serving-cert\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.176100 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.176067 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-trusted-ca-bundle\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.176100 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.176104 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-oauth-serving-cert\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.176336 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.176125 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-config\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.176336 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.176153 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-serving-cert\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.176336 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.176208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-oauth-config\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.176336 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.176242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-service-ca\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.176336 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.176273 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj7vb\" (UniqueName: \"kubernetes.io/projected/6d9e7588-5f55-49fe-9ac6-e59d234c2235-kube-api-access-mj7vb\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.186095 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.185714 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-config\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.186095 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.185739 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-service-ca\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.186095 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.185762 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-oauth-serving-cert\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.186095 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.186066 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-trusted-ca-bundle\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.188132 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.188098 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-oauth-config\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.188363 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.188341 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-serving-cert\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.188452 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.188437 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj7vb\" (UniqueName: \"kubernetes.io/projected/6d9e7588-5f55-49fe-9ac6-e59d234c2235-kube-api-access-mj7vb\") pod \"console-565f9cc7f5-sns9v\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:10.369944 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:10.369898 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:13.914982 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:13.914741 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-565f9cc7f5-sns9v"] Jul 14 13:55:13.917710 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:13.917645 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d9e7588_5f55_49fe_9ac6_e59d234c2235.slice/crio-1fff458321bc4d23279b9b8a438a9ac1f2c924ee745561412a088fdfc827e6fa WatchSource:0}: Error finding container 1fff458321bc4d23279b9b8a438a9ac1f2c924ee745561412a088fdfc827e6fa: Status 404 returned error can't find the container with id 1fff458321bc4d23279b9b8a438a9ac1f2c924ee745561412a088fdfc827e6fa Jul 14 13:55:14.769566 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:14.769524 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdtx6" event={"ID":"2b621f22-17e4-4eba-90e6-b84f76091ee9","Type":"ContainerStarted","Data":"3aa524d4a4e9c2d9d5773ded140ab5ef84ee3a8d71126189d2fac22387c82a46"} Jul 14 13:55:14.771385 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:14.771350 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85kbp" event={"ID":"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a","Type":"ContainerStarted","Data":"ac504b6fa211b8aca9c2d4bc9b5c1c24daeb491f205212c5ca96bddbf937413d"} Jul 14 13:55:14.771493 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:14.771396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-85kbp" event={"ID":"487d9f7a-54b6-4d0b-8b43-da0472dc2a3a","Type":"ContainerStarted","Data":"1faa279357063c316b9d1046a48e3ceab14b3c5e2e5b0684fe6ceaecf0d52b3d"} Jul 14 13:55:14.771493 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:14.771439 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:14.772883 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:14.772860 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22d2x" event={"ID":"983e0528-5495-413e-9bcc-73e699b8e85c","Type":"ContainerStarted","Data":"e8675a746d0b40ff4564097056c583382bc224aac6215bca8b1178e15dcd8c40"} Jul 14 13:55:14.774042 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:14.774021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565f9cc7f5-sns9v" event={"ID":"6d9e7588-5f55-49fe-9ac6-e59d234c2235","Type":"ContainerStarted","Data":"1fff458321bc4d23279b9b8a438a9ac1f2c924ee745561412a088fdfc827e6fa"} Jul 14 13:55:14.792901 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:14.792847 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-85kbp" podStartSLOduration=1.733066451 podStartE2EDuration="7.792831648s" totalImagesPullingTime="6.059765197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:14.791167564 +0000 UTC m=+54.017850506" watchObservedRunningTime="2026-07-14 13:55:14.792831648 +0000 UTC m=+54.019514597" Jul 14 13:55:14.812927 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:14.812881 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-22d2x" podStartSLOduration=1.951758954 podStartE2EDuration="7.81286589s" totalImagesPullingTime="5.861106936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:14.812013095 +0000 UTC m=+54.038696031" watchObservedRunningTime="2026-07-14 13:55:14.81286589 +0000 UTC m=+54.039548807" Jul 14 13:55:15.780948 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:15.780914 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdtx6" event={"ID":"2b621f22-17e4-4eba-90e6-b84f76091ee9","Type":"ContainerStarted","Data":"2748b724c328044a47c0b6ad258e5ae587902c34beed03797af9d917b6b02c30"} Jul 14 13:55:18.053854 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.053816 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vq8th"] Jul 14 13:55:18.055818 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.055801 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.059090 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.059036 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 14 13:55:18.059199 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.059179 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 13:55:18.059823 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.059805 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-g7tp4\"" Jul 14 13:55:18.060569 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.060548 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 13:55:18.060660 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.060549 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 13:55:18.060660 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.060609 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 14 13:55:18.069886 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.069823 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vq8th"] Jul 14 13:55:18.135528 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.135488 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae3900f1-13cb-46fa-ab56-20a295609d26-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.135731 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.135569 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae3900f1-13cb-46fa-ab56-20a295609d26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.135731 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.135595 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph2m5\" (UniqueName: \"kubernetes.io/projected/ae3900f1-13cb-46fa-ab56-20a295609d26-kube-api-access-ph2m5\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.135731 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.135631 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae3900f1-13cb-46fa-ab56-20a295609d26-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.236713 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.236680 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae3900f1-13cb-46fa-ab56-20a295609d26-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.236921 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.236755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae3900f1-13cb-46fa-ab56-20a295609d26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.236921 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.236797 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph2m5\" (UniqueName: \"kubernetes.io/projected/ae3900f1-13cb-46fa-ab56-20a295609d26-kube-api-access-ph2m5\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.236921 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.236844 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae3900f1-13cb-46fa-ab56-20a295609d26-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.237572 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.237547 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae3900f1-13cb-46fa-ab56-20a295609d26-metrics-client-ca\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.240959 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.240927 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae3900f1-13cb-46fa-ab56-20a295609d26-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.240959 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.240949 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae3900f1-13cb-46fa-ab56-20a295609d26-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.247295 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.247261 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph2m5\" (UniqueName: \"kubernetes.io/projected/ae3900f1-13cb-46fa-ab56-20a295609d26-kube-api-access-ph2m5\") pod \"prometheus-operator-64785b6675-vq8th\" (UID: \"ae3900f1-13cb-46fa-ab56-20a295609d26\") " pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.366416 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.366339 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" Jul 14 13:55:18.949629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:18.949597 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-vq8th"] Jul 14 13:55:18.953424 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:18.953400 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3900f1_13cb_46fa_ab56_20a295609d26.slice/crio-59b54e0f8f1a46a4252a18e013f339660445796062062572c435d85af3e6c95d WatchSource:0}: Error finding container 59b54e0f8f1a46a4252a18e013f339660445796062062572c435d85af3e6c95d: Status 404 returned error can't find the container with id 59b54e0f8f1a46a4252a18e013f339660445796062062572c435d85af3e6c95d Jul 14 13:55:19.800595 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:19.800555 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565f9cc7f5-sns9v" event={"ID":"6d9e7588-5f55-49fe-9ac6-e59d234c2235","Type":"ContainerStarted","Data":"f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a"} Jul 14 13:55:19.802991 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:19.802962 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdtx6" event={"ID":"2b621f22-17e4-4eba-90e6-b84f76091ee9","Type":"ContainerStarted","Data":"2e67a50187dbdb091918df27c8caf72b40bc18d1ae5416cb5ef256ad2683de3a"} Jul 14 13:55:19.804198 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:19.804174 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" event={"ID":"ae3900f1-13cb-46fa-ab56-20a295609d26","Type":"ContainerStarted","Data":"59b54e0f8f1a46a4252a18e013f339660445796062062572c435d85af3e6c95d"} Jul 14 13:55:19.820900 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:19.820857 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-565f9cc7f5-sns9v" podStartSLOduration=4.907547458 podStartE2EDuration="9.820843727s" totalImagesPullingTime="4.913296269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:19.820232837 +0000 UTC m=+59.046915751" watchObservedRunningTime="2026-07-14 13:55:19.820843727 +0000 UTC m=+59.047526644" Jul 14 13:55:19.840099 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:19.840051 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mdtx6" podStartSLOduration=1.918864293 podStartE2EDuration="12.840036128s" totalImagesPullingTime="10.921171835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:55:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:19.839350445 +0000 UTC m=+59.066033375" watchObservedRunningTime="2026-07-14 13:55:19.840036128 +0000 UTC m=+59.066719045" Jul 14 13:55:20.370199 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:20.370169 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:20.370386 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:20.370226 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:20.375314 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:20.375276 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:20.808791 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:20.808750 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" event={"ID":"ae3900f1-13cb-46fa-ab56-20a295609d26","Type":"ContainerStarted","Data":"af66b64491f94909cce1ba42c563cbff0b71e0f03c5e7fccc7bf9fbfe4b38e07"} Jul 14 13:55:20.809217 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:20.808809 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" event={"ID":"ae3900f1-13cb-46fa-ab56-20a295609d26","Type":"ContainerStarted","Data":"57169a0ae511f9dfa93c7885abf4c0a2c956cb5e627dfa31a640d3013ad8bdc8"} Jul 14 13:55:20.812733 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:20.812708 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:55:20.827645 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:20.827603 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-vq8th" podStartSLOduration=1.568910743 podStartE2EDuration="2.827590161s" totalImagesPullingTime="1.258679418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:20.826052589 +0000 UTC m=+60.052735530" watchObservedRunningTime="2026-07-14 13:55:20.827590161 +0000 UTC m=+60.054273077" Jul 14 13:55:22.418268 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.418236 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c"] Jul 14 13:55:22.422269 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.422253 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.424854 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.424836 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-kl4lc\"" Jul 14 13:55:22.425028 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.425009 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 14 13:55:22.425123 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.425109 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 14 13:55:22.434161 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.434143 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c"] Jul 14 13:55:22.459520 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.459487 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kwwdb"] Jul 14 13:55:22.463202 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.463181 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.465809 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.465787 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 13:55:22.465940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.465815 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-krlv4\"" Jul 14 13:55:22.465940 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.465832 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 13:55:22.466403 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.466390 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 13:55:22.470918 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.470899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.471017 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.470953 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1489a730-84bc-4242-b530-bf640674402b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.471066 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.471033 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.471102 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.471069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44wnm\" (UniqueName: \"kubernetes.io/projected/1489a730-84bc-4242-b530-bf640674402b-kube-api-access-44wnm\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.571582 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571541 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571582 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-root\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571778 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571615 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1489a730-84bc-4242-b530-bf640674402b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.571778 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571637 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-sys\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571778 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571660 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-tls\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571778 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571681 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44wnm\" (UniqueName: \"kubernetes.io/projected/1489a730-84bc-4242-b530-bf640674402b-kube-api-access-44wnm\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.571778 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-accelerators-collector-config\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571778 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.571993 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571797 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-wtmp\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571993 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tzhp\" (UniqueName: \"kubernetes.io/projected/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-kube-api-access-9tzhp\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571993 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-textfile\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571993 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-metrics-client-ca\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.571993 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.571927 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.572153 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:22.572019 2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 14 13:55:22.572153 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:22.572068 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-tls podName:1489a730-84bc-4242-b530-bf640674402b nodeName:}" failed. No retries permitted until 2026-07-14 13:55:23.072054324 +0000 UTC m=+62.298737222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-xhf2c" (UID: "1489a730-84bc-4242-b530-bf640674402b") : secret "openshift-state-metrics-tls" not found Jul 14 13:55:22.572318 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.572300 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1489a730-84bc-4242-b530-bf640674402b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.574079 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.574059 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.582951 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.582919 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44wnm\" (UniqueName: \"kubernetes.io/projected/1489a730-84bc-4242-b530-bf640674402b-kube-api-access-44wnm\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:22.672987 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.672905 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-sys\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.672987 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.672945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-tls\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.672987 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.672968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-accelerators-collector-config\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673197 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-sys\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673197 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-wtmp\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673197 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673128 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tzhp\" (UniqueName: \"kubernetes.io/projected/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-kube-api-access-9tzhp\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673197 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-textfile\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673197 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673190 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-metrics-client-ca\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673448 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673448 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673302 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-root\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673448 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673314 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-wtmp\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673448 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673372 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-root\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673516 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-textfile\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-accelerators-collector-config\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.673813 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.673792 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-metrics-client-ca\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.675307 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.675273 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-tls\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.675503 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.675485 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.681369 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.681348 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tzhp\" (UniqueName: \"kubernetes.io/projected/74f7618b-5c80-40e6-9fee-dc7705cd7dd6-kube-api-access-9tzhp\") pod \"node-exporter-kwwdb\" (UID: \"74f7618b-5c80-40e6-9fee-dc7705cd7dd6\") " pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.715908 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.715879 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kspwb" Jul 14 13:55:22.772496 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.772468 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kwwdb" Jul 14 13:55:22.803320 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:22.803300 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74f7618b_5c80_40e6_9fee_dc7705cd7dd6.slice/crio-bee46825b43543d06b904b79715fa316bf3032d41050141148488bafb6b009c4 WatchSource:0}: Error finding container bee46825b43543d06b904b79715fa316bf3032d41050141148488bafb6b009c4: Status 404 returned error can't find the container with id bee46825b43543d06b904b79715fa316bf3032d41050141148488bafb6b009c4 Jul 14 13:55:22.816016 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:22.815987 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwwdb" event={"ID":"74f7618b-5c80-40e6-9fee-dc7705cd7dd6","Type":"ContainerStarted","Data":"bee46825b43543d06b904b79715fa316bf3032d41050141148488bafb6b009c4"} Jul 14 13:55:23.076829 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.076794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:23.079118 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.079099 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1489a730-84bc-4242-b530-bf640674402b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-xhf2c\" (UID: \"1489a730-84bc-4242-b530-bf640674402b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:23.331654 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.331569 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" Jul 14 13:55:23.487049 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.487016 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:55:23.495708 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:23.495680 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1489a730_84bc_4242_b530_bf640674402b.slice/crio-9fe2209fac65d5e18f040bb595603d9ff0e1eb7f2a290c453b9003cd58faf273 WatchSource:0}: Error finding container 9fe2209fac65d5e18f040bb595603d9ff0e1eb7f2a290c453b9003cd58faf273: Status 404 returned error can't find the container with id 9fe2209fac65d5e18f040bb595603d9ff0e1eb7f2a290c453b9003cd58faf273 Jul 14 13:55:23.515446 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.515421 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c"] Jul 14 13:55:23.515576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.515460 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:55:23.515576 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.515570 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.518585 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518564 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 13:55:23.518585 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518574 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 13:55:23.518774 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518574 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qczld\"" Jul 14 13:55:23.518774 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518628 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 13:55:23.518774 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518575 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 13:55:23.518774 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518623 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 13:55:23.518774 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518574 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 13:55:23.519021 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518806 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 13:55:23.519021 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518824 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 13:55:23.519021 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.518851 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 13:55:23.581630 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581603 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.581812 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43215d03-c9bb-45e1-81f3-a0abb88a37e3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.581812 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581692 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6qkj\" (UniqueName: \"kubernetes.io/projected/43215d03-c9bb-45e1-81f3-a0abb88a37e3-kube-api-access-j6qkj\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.581812 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581740 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43215d03-c9bb-45e1-81f3-a0abb88a37e3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.581812 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581773 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43215d03-c9bb-45e1-81f3-a0abb88a37e3-config-out\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.581812 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581797 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43215d03-c9bb-45e1-81f3-a0abb88a37e3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.582063 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581824 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.582063 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581900 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43215d03-c9bb-45e1-81f3-a0abb88a37e3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.582063 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.581980 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-web-config\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.582063 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.582036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-config-volume\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.582063 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.582060 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.582314 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.582114 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.582314 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.582172 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.683507 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43215d03-c9bb-45e1-81f3-a0abb88a37e3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.683636 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683516 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6qkj\" (UniqueName: \"kubernetes.io/projected/43215d03-c9bb-45e1-81f3-a0abb88a37e3-kube-api-access-j6qkj\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.683636 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683605 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43215d03-c9bb-45e1-81f3-a0abb88a37e3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.683748 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683640 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43215d03-c9bb-45e1-81f3-a0abb88a37e3-config-out\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.683748 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683664 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43215d03-c9bb-45e1-81f3-a0abb88a37e3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.683852 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683818 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.683907 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43215d03-c9bb-45e1-81f3-a0abb88a37e3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.683963 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-web-config\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.684014 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.683992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-config-volume\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.684067 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.684016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.684067 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.684062 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.684179 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.684069 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43215d03-c9bb-45e1-81f3-a0abb88a37e3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.684179 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.684096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.684179 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.684129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.684895 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.684710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43215d03-c9bb-45e1-81f3-a0abb88a37e3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.684895 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:23.684804 2564 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 14 13:55:23.685455 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:23.685091 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-main-tls podName:43215d03-c9bb-45e1-81f3-a0abb88a37e3 nodeName:}" failed. No retries permitted until 2026-07-14 13:55:24.185068666 +0000 UTC m=+63.411751575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "43215d03-c9bb-45e1-81f3-a0abb88a37e3") : secret "alertmanager-main-tls" not found Jul 14 13:55:23.685455 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.685177 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43215d03-c9bb-45e1-81f3-a0abb88a37e3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.686969 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.686896 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.686969 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.686932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-config-volume\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.687220 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.687173 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43215d03-c9bb-45e1-81f3-a0abb88a37e3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.687340 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.687258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.687909 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.687887 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.687986 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.687894 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43215d03-c9bb-45e1-81f3-a0abb88a37e3-config-out\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.688140 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.688120 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-web-config\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.688236 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.688220 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.696880 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.696857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6qkj\" (UniqueName: \"kubernetes.io/projected/43215d03-c9bb-45e1-81f3-a0abb88a37e3-kube-api-access-j6qkj\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:23.822569 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.822532 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" event={"ID":"1489a730-84bc-4242-b530-bf640674402b","Type":"ContainerStarted","Data":"d3f7ccfdaaf5b33106c07c93d1bb859e3253eb75e527886777e349dc5fd96753"} Jul 14 13:55:23.822722 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.822592 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" event={"ID":"1489a730-84bc-4242-b530-bf640674402b","Type":"ContainerStarted","Data":"12d4f0791b16f7bfe2f9538ab11544a9ae007375a999cb498d108159c96dc24a"} Jul 14 13:55:23.822722 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:23.822617 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" event={"ID":"1489a730-84bc-4242-b530-bf640674402b","Type":"ContainerStarted","Data":"9fe2209fac65d5e18f040bb595603d9ff0e1eb7f2a290c453b9003cd58faf273"} Jul 14 13:55:24.189178 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:24.189139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:24.191727 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:24.191701 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43215d03-c9bb-45e1-81f3-a0abb88a37e3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43215d03-c9bb-45e1-81f3-a0abb88a37e3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:24.425205 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:24.425162 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 13:55:24.579178 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:24.579146 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 13:55:24.582058 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:24.582026 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43215d03_c9bb_45e1_81f3_a0abb88a37e3.slice/crio-46178dada2695156dcfe18716c4aee7c29f5807fd83a006c8357a6a06ebc9c60 WatchSource:0}: Error finding container 46178dada2695156dcfe18716c4aee7c29f5807fd83a006c8357a6a06ebc9c60: Status 404 returned error can't find the container with id 46178dada2695156dcfe18716c4aee7c29f5807fd83a006c8357a6a06ebc9c60 Jul 14 13:55:24.783440 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:24.783355 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-85kbp" Jul 14 13:55:24.828879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:24.828712 2564 generic.go:356] "Generic (PLEG): container finished" podID="74f7618b-5c80-40e6-9fee-dc7705cd7dd6" containerID="3f82224e0c4b1368af9b1777703d41c057d1fa65318789901d72e8244d4c400b" exitCode=0 Jul 14 13:55:24.828879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:24.828817 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwwdb" event={"ID":"74f7618b-5c80-40e6-9fee-dc7705cd7dd6","Type":"ContainerDied","Data":"3f82224e0c4b1368af9b1777703d41c057d1fa65318789901d72e8244d4c400b"} Jul 14 13:55:24.830415 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:24.830390 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43215d03-c9bb-45e1-81f3-a0abb88a37e3","Type":"ContainerStarted","Data":"46178dada2695156dcfe18716c4aee7c29f5807fd83a006c8357a6a06ebc9c60"} Jul 14 13:55:25.837598 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:25.837562 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" event={"ID":"1489a730-84bc-4242-b530-bf640674402b","Type":"ContainerStarted","Data":"4814c93e01182ea1f8c95440ae4025e698926a62ee6b123127eb0e9a95101c5d"} Jul 14 13:55:25.839634 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:25.839611 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwwdb" event={"ID":"74f7618b-5c80-40e6-9fee-dc7705cd7dd6","Type":"ContainerStarted","Data":"5223f4f7bcf8cfe00c97d4e6040c62da94be2cecf96f776ee6ad0b6dab7aaaf3"} Jul 14 13:55:25.839758 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:25.839657 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kwwdb" event={"ID":"74f7618b-5c80-40e6-9fee-dc7705cd7dd6","Type":"ContainerStarted","Data":"9bf0608b41c067bdd36ab9798fadb85588a9b68c920994cacce83930c61709cb"} Jul 14 13:55:25.858597 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:25.858550 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-xhf2c" podStartSLOduration=2.6848460530000002 podStartE2EDuration="3.85853597s" totalImagesPullingTime="1.173689917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:25.856468555 +0000 UTC m=+65.083151472" watchObservedRunningTime="2026-07-14 13:55:25.85853597 +0000 UTC m=+65.085218887" Jul 14 13:55:25.879743 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:25.879704 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kwwdb" podStartSLOduration=2.393855936 podStartE2EDuration="3.879688754s" totalImagesPullingTime="1.421291683s" totalInitContainerRuntime="64.541135ms" isStatefulPod=true podCreationTimestamp="2026-07-14 13:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:25.878666833 +0000 UTC m=+65.105349751" watchObservedRunningTime="2026-07-14 13:55:25.879688754 +0000 UTC m=+65.106371671" Jul 14 13:55:26.824608 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.824576 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-595cf47db7-fhfld"] Jul 14 13:55:26.826801 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.826782 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:26.829272 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.829247 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 14 13:55:26.829411 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.829334 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 14 13:55:26.829487 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.829426 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 14 13:55:26.829487 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.829434 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 13:55:26.830896 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.830875 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-sxk68\"" Jul 14 13:55:26.830998 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.830975 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-n9c697s8r5ph\"" Jul 14 13:55:26.837768 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.837749 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-595cf47db7-fhfld"] Jul 14 13:55:26.844526 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.844504 2564 generic.go:356] "Generic (PLEG): container finished" podID="43215d03-c9bb-45e1-81f3-a0abb88a37e3" containerID="1cf3f17696d9e6e8322790688491ced6e4d7445e73fe005367c5d1fe55de91c0" exitCode=0 Jul 14 13:55:26.844634 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.844581 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43215d03-c9bb-45e1-81f3-a0abb88a37e3","Type":"ContainerDied","Data":"1cf3f17696d9e6e8322790688491ced6e4d7445e73fe005367c5d1fe55de91c0"} Jul 14 13:55:26.911575 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.911537 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-secret-metrics-server-client-certs\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:26.911746 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.911597 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/969f0d24-40f8-409e-9af3-15913ccba965-metrics-server-audit-profiles\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:26.911746 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.911683 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969f0d24-40f8-409e-9af3-15913ccba965-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:26.912025 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.911995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-secret-metrics-server-tls\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:26.912122 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.912031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-client-ca-bundle\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:26.912213 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.912192 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpk9d\" (UniqueName: \"kubernetes.io/projected/969f0d24-40f8-409e-9af3-15913ccba965-kube-api-access-rpk9d\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:26.912307 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:26.912273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/969f0d24-40f8-409e-9af3-15913ccba965-audit-log\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.012843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.012799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-secret-metrics-server-tls\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.012843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.012840 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-client-ca-bundle\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.013082 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.012862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:55:27.013082 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.012892 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpk9d\" (UniqueName: \"kubernetes.io/projected/969f0d24-40f8-409e-9af3-15913ccba965-kube-api-access-rpk9d\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.013082 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.012923 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/969f0d24-40f8-409e-9af3-15913ccba965-audit-log\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.013082 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.012959 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-secret-metrics-server-client-certs\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.013082 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.012996 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/969f0d24-40f8-409e-9af3-15913ccba965-metrics-server-audit-profiles\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.013082 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.013036 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969f0d24-40f8-409e-9af3-15913ccba965-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.013516 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.013487 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/969f0d24-40f8-409e-9af3-15913ccba965-audit-log\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.013881 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.013859 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/969f0d24-40f8-409e-9af3-15913ccba965-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.014120 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.014098 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/969f0d24-40f8-409e-9af3-15913ccba965-metrics-server-audit-profiles\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.015374 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.015356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-client-ca-bundle\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.015458 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.015366 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-secret-metrics-server-tls\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.015563 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.015544 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/969f0d24-40f8-409e-9af3-15913ccba965-secret-metrics-server-client-certs\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.015663 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.015647 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 13:55:27.020580 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.020563 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpk9d\" (UniqueName: \"kubernetes.io/projected/969f0d24-40f8-409e-9af3-15913ccba965-kube-api-access-rpk9d\") pod \"metrics-server-595cf47db7-fhfld\" (UID: \"969f0d24-40f8-409e-9af3-15913ccba965\") " pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.024968 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.024951 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e4f6a09e-45c4-4875-908a-18f27e9c527c-original-pull-secret\") pod \"global-pull-secret-syncer-w8zzc\" (UID: \"e4f6a09e-45c4-4875-908a-18f27e9c527c\") " pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:55:27.114492 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.114405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:55:27.117001 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.116982 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 13:55:27.127220 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.127198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4490358-1756-4a8e-b8a6-d2503b9fdf31-metrics-certs\") pod \"network-metrics-daemon-zrcm7\" (UID: \"b4490358-1756-4a8e-b8a6-d2503b9fdf31\") " pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:55:27.136314 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.136275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:27.175256 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.175226 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w8zzc" Jul 14 13:55:27.187498 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.187476 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v5knd\"" Jul 14 13:55:27.195611 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.195564 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zrcm7" Jul 14 13:55:27.210237 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.210140 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl"] Jul 14 13:55:27.213152 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.213129 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:27.215926 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.215634 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:27.215926 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.215770 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 14 13:55:27.215926 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.215867 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-tcgqg\"" Jul 14 13:55:27.217993 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.217936 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 13:55:27.224638 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.224569 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79ccdbc5dc-gs5bm"] Jul 14 13:55:27.231066 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.231047 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl"] Jul 14 13:55:27.231349 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.231174 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.243988 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.238302 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 13:55:27.243988 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.241222 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79ccdbc5dc-gs5bm"] Jul 14 13:55:27.247223 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.247166 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq6sl\" (UniqueName: \"kubernetes.io/projected/a5fa31a2-e916-4f6b-b0d7-4653f7f89d16-kube-api-access-bq6sl\") pod \"network-check-target-gz22v\" (UID: \"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16\") " pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:27.296993 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.296962 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-595cf47db7-fhfld"] Jul 14 13:55:27.300055 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:27.300022 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969f0d24_40f8_409e_9af3_15913ccba965.slice/crio-e2bfb76bbace87d4cccdca63e0ff2bda58323501bfe0ce9ebb7a30926f90750f WatchSource:0}: Error finding container e2bfb76bbace87d4cccdca63e0ff2bda58323501bfe0ce9ebb7a30926f90750f: Status 404 returned error can't find the container with id e2bfb76bbace87d4cccdca63e0ff2bda58323501bfe0ce9ebb7a30926f90750f Jul 14 13:55:27.317027 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.316957 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-oauth-config\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.317181 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.317036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-console-config\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.317181 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.317083 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-serving-cert\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.317181 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.317112 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzwsj\" (UniqueName: \"kubernetes.io/projected/4a1c3119-30ff-422a-bdcc-a576fce3b521-kube-api-access-gzwsj\") pod \"monitoring-plugin-5f68df7f58-mxhgl\" (UID: \"4a1c3119-30ff-422a-bdcc-a576fce3b521\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:27.317181 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.317145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-oauth-serving-cert\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.317181 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.317174 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8thvp\" (UniqueName: \"kubernetes.io/projected/cc0a2e29-e340-44a1-a425-f08fb8315616-kube-api-access-8thvp\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.317511 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.317199 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-trusted-ca-bundle\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.317511 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.317247 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4a1c3119-30ff-422a-bdcc-a576fce3b521-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mxhgl\" (UID: \"4a1c3119-30ff-422a-bdcc-a576fce3b521\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:27.317511 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.317313 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-service-ca\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.343191 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.343036 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w8zzc"] Jul 14 13:55:27.345765 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:27.345740 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f6a09e_45c4_4875_908a_18f27e9c527c.slice/crio-513d84bd39eb0c66bd7908e1ec6881a836c273fd5b3e8952f92b81ee078dff5d WatchSource:0}: Error finding container 513d84bd39eb0c66bd7908e1ec6881a836c273fd5b3e8952f92b81ee078dff5d: Status 404 returned error can't find the container with id 513d84bd39eb0c66bd7908e1ec6881a836c273fd5b3e8952f92b81ee078dff5d Jul 14 13:55:27.356671 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.356648 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zrcm7"] Jul 14 13:55:27.359501 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:27.359475 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4490358_1756_4a8e_b8a6_d2503b9fdf31.slice/crio-e7c68b80f910289a20c0c268b635128a124d3b7389988d00e9a5f203711c2171 WatchSource:0}: Error finding container e7c68b80f910289a20c0c268b635128a124d3b7389988d00e9a5f203711c2171: Status 404 returned error can't find the container with id e7c68b80f910289a20c0c268b635128a124d3b7389988d00e9a5f203711c2171 Jul 14 13:55:27.418443 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418345 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-oauth-serving-cert\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.418443 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418389 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8thvp\" (UniqueName: \"kubernetes.io/projected/cc0a2e29-e340-44a1-a425-f08fb8315616-kube-api-access-8thvp\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.418443 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418422 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-trusted-ca-bundle\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.418704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4a1c3119-30ff-422a-bdcc-a576fce3b521-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mxhgl\" (UID: \"4a1c3119-30ff-422a-bdcc-a576fce3b521\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:27.418704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-service-ca\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.418704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418537 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-oauth-config\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.418704 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418563 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-console-config\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.418704 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:27.418604 2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 14 13:55:27.418704 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:55:27.418680 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a1c3119-30ff-422a-bdcc-a576fce3b521-monitoring-plugin-cert podName:4a1c3119-30ff-422a-bdcc-a576fce3b521 nodeName:}" failed. No retries permitted until 2026-07-14 13:55:27.918658228 +0000 UTC m=+67.145341140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/4a1c3119-30ff-422a-bdcc-a576fce3b521-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-mxhgl" (UID: "4a1c3119-30ff-422a-bdcc-a576fce3b521") : secret "monitoring-plugin-cert" not found Jul 14 13:55:27.419010 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418605 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-serving-cert\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.419010 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.418780 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzwsj\" (UniqueName: \"kubernetes.io/projected/4a1c3119-30ff-422a-bdcc-a576fce3b521-kube-api-access-gzwsj\") pod \"monitoring-plugin-5f68df7f58-mxhgl\" (UID: \"4a1c3119-30ff-422a-bdcc-a576fce3b521\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:27.419164 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.419137 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-oauth-serving-cert\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.419421 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.419402 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-service-ca\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.419496 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.419483 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-console-config\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.419787 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.419771 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-trusted-ca-bundle\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.421048 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.421032 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-serving-cert\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.421101 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.421047 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-oauth-config\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.427311 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.427269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzwsj\" (UniqueName: \"kubernetes.io/projected/4a1c3119-30ff-422a-bdcc-a576fce3b521-kube-api-access-gzwsj\") pod \"monitoring-plugin-5f68df7f58-mxhgl\" (UID: \"4a1c3119-30ff-422a-bdcc-a576fce3b521\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:27.427826 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.427810 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8thvp\" (UniqueName: \"kubernetes.io/projected/cc0a2e29-e340-44a1-a425-f08fb8315616-kube-api-access-8thvp\") pod \"console-79ccdbc5dc-gs5bm\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.483427 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.483403 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7c4p6\"" Jul 14 13:55:27.490939 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.490918 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:27.556863 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.556838 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:27.622271 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.622236 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gz22v"] Jul 14 13:55:27.626195 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:27.626166 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5fa31a2_e916_4f6b_b0d7_4653f7f89d16.slice/crio-f502438413a1642980b51508ef6a420945011cfae4a59818c47026c1bc1826ea WatchSource:0}: Error finding container f502438413a1642980b51508ef6a420945011cfae4a59818c47026c1bc1826ea: Status 404 returned error can't find the container with id f502438413a1642980b51508ef6a420945011cfae4a59818c47026c1bc1826ea Jul 14 13:55:27.689239 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.689154 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79ccdbc5dc-gs5bm"] Jul 14 13:55:27.692986 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:27.692960 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0a2e29_e340_44a1_a425_f08fb8315616.slice/crio-2f49210dea0099561634cef482f5347ba16037b9b180c4167e864aebc6a04775 WatchSource:0}: Error finding container 2f49210dea0099561634cef482f5347ba16037b9b180c4167e864aebc6a04775: Status 404 returned error can't find the container with id 2f49210dea0099561634cef482f5347ba16037b9b180c4167e864aebc6a04775 Jul 14 13:55:27.849663 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.849623 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" event={"ID":"969f0d24-40f8-409e-9af3-15913ccba965","Type":"ContainerStarted","Data":"e2bfb76bbace87d4cccdca63e0ff2bda58323501bfe0ce9ebb7a30926f90750f"} Jul 14 13:55:27.851486 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.851445 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79ccdbc5dc-gs5bm" event={"ID":"cc0a2e29-e340-44a1-a425-f08fb8315616","Type":"ContainerStarted","Data":"5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef"} Jul 14 13:55:27.851627 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.851521 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79ccdbc5dc-gs5bm" event={"ID":"cc0a2e29-e340-44a1-a425-f08fb8315616","Type":"ContainerStarted","Data":"2f49210dea0099561634cef482f5347ba16037b9b180c4167e864aebc6a04775"} Jul 14 13:55:27.852777 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.852748 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gz22v" event={"ID":"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16","Type":"ContainerStarted","Data":"f502438413a1642980b51508ef6a420945011cfae4a59818c47026c1bc1826ea"} Jul 14 13:55:27.853983 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.853961 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zrcm7" event={"ID":"b4490358-1756-4a8e-b8a6-d2503b9fdf31","Type":"ContainerStarted","Data":"e7c68b80f910289a20c0c268b635128a124d3b7389988d00e9a5f203711c2171"} Jul 14 13:55:27.855092 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.855070 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w8zzc" event={"ID":"e4f6a09e-45c4-4875-908a-18f27e9c527c","Type":"ContainerStarted","Data":"513d84bd39eb0c66bd7908e1ec6881a836c273fd5b3e8952f92b81ee078dff5d"} Jul 14 13:55:27.871055 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.870998 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79ccdbc5dc-gs5bm" podStartSLOduration=0.870972784 podStartE2EDuration="870.972784ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:27.869846418 +0000 UTC m=+67.096529336" watchObservedRunningTime="2026-07-14 13:55:27.870972784 +0000 UTC m=+67.097655702" Jul 14 13:55:27.923484 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.923446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4a1c3119-30ff-422a-bdcc-a576fce3b521-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mxhgl\" (UID: \"4a1c3119-30ff-422a-bdcc-a576fce3b521\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:27.927094 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:27.926898 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4a1c3119-30ff-422a-bdcc-a576fce3b521-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mxhgl\" (UID: \"4a1c3119-30ff-422a-bdcc-a576fce3b521\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:28.124625 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:28.124327 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:28.304910 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:28.304877 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl"] Jul 14 13:55:28.568182 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:55:28.568135 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a1c3119_30ff_422a_bdcc_a576fce3b521.slice/crio-b1f07fc8d7729b5e218033f44b7e319b123ec1dbfd1a88db2bcf71cf4e7d6ffe WatchSource:0}: Error finding container b1f07fc8d7729b5e218033f44b7e319b123ec1dbfd1a88db2bcf71cf4e7d6ffe: Status 404 returned error can't find the container with id b1f07fc8d7729b5e218033f44b7e319b123ec1dbfd1a88db2bcf71cf4e7d6ffe Jul 14 13:55:28.864751 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:28.864664 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" event={"ID":"4a1c3119-30ff-422a-bdcc-a576fce3b521","Type":"ContainerStarted","Data":"b1f07fc8d7729b5e218033f44b7e319b123ec1dbfd1a88db2bcf71cf4e7d6ffe"} Jul 14 13:55:30.873896 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:30.873860 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zrcm7" event={"ID":"b4490358-1756-4a8e-b8a6-d2503b9fdf31","Type":"ContainerStarted","Data":"0fb88f20fcb30d908d7a1397cd811d50fe3e4d342bef62239743803bf496828d"} Jul 14 13:55:30.875590 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:30.875545 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" event={"ID":"969f0d24-40f8-409e-9af3-15913ccba965","Type":"ContainerStarted","Data":"8ecefe3e4b92cb367c1cf3ab00903865ee9ce3216a2759d35d1144d25e42a332"} Jul 14 13:55:30.893983 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:30.893926 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" podStartSLOduration=2.185579209 podStartE2EDuration="4.893909163s" totalImagesPullingTime="2.708329954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:30.892860777 +0000 UTC m=+70.119543696" watchObservedRunningTime="2026-07-14 13:55:30.893909163 +0000 UTC m=+70.120592076" Jul 14 13:55:33.889970 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.889874 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zrcm7" event={"ID":"b4490358-1756-4a8e-b8a6-d2503b9fdf31","Type":"ContainerStarted","Data":"e372651ac5cd5ec2adcdfe3b2c712ad4eb008c7b3a434ce27255a845699c5f72"} Jul 14 13:55:33.891396 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.891367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w8zzc" event={"ID":"e4f6a09e-45c4-4875-908a-18f27e9c527c","Type":"ContainerStarted","Data":"016a020403c5d0c486f081aa225dbf43125f11035c1e249597172c371834839c"} Jul 14 13:55:33.894313 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.894266 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43215d03-c9bb-45e1-81f3-a0abb88a37e3","Type":"ContainerStarted","Data":"94021125d38cd5f4d2cb07c24e14d141a67498cf8654666a93b6dacf2ea3cc13"} Jul 14 13:55:33.894431 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.894338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43215d03-c9bb-45e1-81f3-a0abb88a37e3","Type":"ContainerStarted","Data":"37061a69488fc6197c5f44db3ebf373202e27c5f24117de66ff345a454a0ff8b"} Jul 14 13:55:33.894431 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.894365 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43215d03-c9bb-45e1-81f3-a0abb88a37e3","Type":"ContainerStarted","Data":"81c97006d14c9c13dbfabab6acbeea1f74f5f6a5d666c9ff785d6a9021d2c4b9"} Jul 14 13:55:33.894431 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.894396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43215d03-c9bb-45e1-81f3-a0abb88a37e3","Type":"ContainerStarted","Data":"df5683219b4cacd98d868a47d697c96be165f6a2de2f4041cc647bcb6be2bfcc"} Jul 14 13:55:33.894431 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.894423 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43215d03-c9bb-45e1-81f3-a0abb88a37e3","Type":"ContainerStarted","Data":"fe66b27a3ef5b81e1203d7588d10b9cde57988bc7ea6124b247b744500f9602a"} Jul 14 13:55:33.895696 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.895673 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" event={"ID":"4a1c3119-30ff-422a-bdcc-a576fce3b521","Type":"ContainerStarted","Data":"5508ffa49d7abf1ec9249966dc44f4ba589e4a1d54502835caea53ab2d2728f3"} Jul 14 13:55:33.895880 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.895862 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:33.897140 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.897118 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gz22v" event={"ID":"a5fa31a2-e916-4f6b-b0d7-4653f7f89d16","Type":"ContainerStarted","Data":"a2b67964dff1bfcaf788901f3e9acc2e22c91f3b5c5e4b8b9c1d47315325c7a8"} Jul 14 13:55:33.897260 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.897246 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:55:33.900908 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.900881 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" Jul 14 13:55:33.907165 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.907118 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zrcm7" podStartSLOduration=70.261392158 podStartE2EDuration="1m12.90710283s" totalImagesPullingTime="2.645710672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:33.906534982 +0000 UTC m=+73.133217917" watchObservedRunningTime="2026-07-14 13:55:33.90710283 +0000 UTC m=+73.133785742" Jul 14 13:55:33.923558 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.923514 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gz22v" podStartSLOduration=67.206449358 podStartE2EDuration="1m12.923500415s" totalImagesPullingTime="5.717051057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:33.92249899 +0000 UTC m=+73.149181936" watchObservedRunningTime="2026-07-14 13:55:33.923500415 +0000 UTC m=+73.150183331" Jul 14 13:55:33.940763 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.940708 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mxhgl" podStartSLOduration=2.176659832 podStartE2EDuration="6.940693593s" totalImagesPullingTime="4.764033761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:33.939593439 +0000 UTC m=+73.166276387" watchObservedRunningTime="2026-07-14 13:55:33.940693593 +0000 UTC m=+73.167376509" Jul 14 13:55:33.955993 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:33.955946 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w8zzc" podStartSLOduration=66.954064863 podStartE2EDuration="1m12.955931184s" totalImagesPullingTime="6.001866321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 13:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:33.955257191 +0000 UTC m=+73.181940110" watchObservedRunningTime="2026-07-14 13:55:33.955931184 +0000 UTC m=+73.182614100" Jul 14 13:55:34.904824 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:34.904780 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43215d03-c9bb-45e1-81f3-a0abb88a37e3","Type":"ContainerStarted","Data":"96d8799d810b1aa18aae1ea3f187c78ba462c07387c8c25adc22ddcc483ebabb"} Jul 14 13:55:34.937138 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:34.937075 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.423172112 podStartE2EDuration="11.937051468s" totalImagesPullingTime="8.498572615s" totalInitContainerRuntime="15.306741ms" isStatefulPod=false podCreationTimestamp="2026-07-14 13:55:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:55:34.936170413 +0000 UTC m=+74.162853329" watchObservedRunningTime="2026-07-14 13:55:34.937051468 +0000 UTC m=+74.163734387" Jul 14 13:55:37.557076 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:37.557033 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:37.557510 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:37.557109 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:37.562728 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:37.562704 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:37.922394 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:37.922307 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:55:37.988020 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:37.987988 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-565f9cc7f5-sns9v"] Jul 14 13:55:47.137236 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:47.137099 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:55:47.137236 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:55:47.137163 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:56:03.014436 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.014368 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-565f9cc7f5-sns9v" podUID="6d9e7588-5f55-49fe-9ac6-e59d234c2235" containerName="console" containerID="cri-o://f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a" gracePeriod=15 Jul 14 13:56:03.255183 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.255160 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:56:03.418513 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.418476 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mj7vb\" (UniqueName: \"kubernetes.io/projected/6d9e7588-5f55-49fe-9ac6-e59d234c2235-kube-api-access-mj7vb\") pod \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " Jul 14 13:56:03.418686 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.418534 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-config\") pod \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " Jul 14 13:56:03.418686 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.418565 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-oauth-config\") pod \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " Jul 14 13:56:03.418686 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.418593 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-trusted-ca-bundle\") pod \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " Jul 14 13:56:03.418686 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.418613 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-service-ca\") pod \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " Jul 14 13:56:03.418686 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.418637 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-oauth-serving-cert\") pod \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " Jul 14 13:56:03.418686 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.418654 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-serving-cert\") pod \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\" (UID: \"6d9e7588-5f55-49fe-9ac6-e59d234c2235\") " Jul 14 13:56:03.419116 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.419078 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-config" (OuterVolumeSpecName: "console-config") pod "6d9e7588-5f55-49fe-9ac6-e59d234c2235" (UID: "6d9e7588-5f55-49fe-9ac6-e59d234c2235"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:56:03.419116 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.419094 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6d9e7588-5f55-49fe-9ac6-e59d234c2235" (UID: "6d9e7588-5f55-49fe-9ac6-e59d234c2235"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:56:03.419116 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.419083 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-service-ca" (OuterVolumeSpecName: "service-ca") pod "6d9e7588-5f55-49fe-9ac6-e59d234c2235" (UID: "6d9e7588-5f55-49fe-9ac6-e59d234c2235"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:56:03.419302 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.419161 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6d9e7588-5f55-49fe-9ac6-e59d234c2235" (UID: "6d9e7588-5f55-49fe-9ac6-e59d234c2235"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:56:03.421117 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.421086 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6d9e7588-5f55-49fe-9ac6-e59d234c2235" (UID: "6d9e7588-5f55-49fe-9ac6-e59d234c2235"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:56:03.421198 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.421159 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6d9e7588-5f55-49fe-9ac6-e59d234c2235" (UID: "6d9e7588-5f55-49fe-9ac6-e59d234c2235"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:56:03.421198 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.421159 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d9e7588-5f55-49fe-9ac6-e59d234c2235-kube-api-access-mj7vb" (OuterVolumeSpecName: "kube-api-access-mj7vb") pod "6d9e7588-5f55-49fe-9ac6-e59d234c2235" (UID: "6d9e7588-5f55-49fe-9ac6-e59d234c2235"). InnerVolumeSpecName "kube-api-access-mj7vb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:56:03.519901 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.519870 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:56:03.519901 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.519897 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-oauth-serving-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:56:03.519901 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.519906 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-serving-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:56:03.520194 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.519917 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj7vb\" (UniqueName: \"kubernetes.io/projected/6d9e7588-5f55-49fe-9ac6-e59d234c2235-kube-api-access-mj7vb\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:56:03.520194 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.519926 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-config\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:56:03.520194 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.519934 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d9e7588-5f55-49fe-9ac6-e59d234c2235-console-oauth-config\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:56:03.520194 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:03.519942 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d9e7588-5f55-49fe-9ac6-e59d234c2235-trusted-ca-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:56:04.013212 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.013181 2564 generic.go:356] "Generic (PLEG): container finished" podID="6d9e7588-5f55-49fe-9ac6-e59d234c2235" containerID="f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a" exitCode=0 Jul 14 13:56:04.013411 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.013227 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-565f9cc7f5-sns9v" Jul 14 13:56:04.013411 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.013260 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565f9cc7f5-sns9v" event={"ID":"6d9e7588-5f55-49fe-9ac6-e59d234c2235","Type":"ContainerDied","Data":"f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a"} Jul 14 13:56:04.013411 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.013329 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-565f9cc7f5-sns9v" event={"ID":"6d9e7588-5f55-49fe-9ac6-e59d234c2235","Type":"ContainerDied","Data":"1fff458321bc4d23279b9b8a438a9ac1f2c924ee745561412a088fdfc827e6fa"} Jul 14 13:56:04.013411 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.013353 2564 scope.go:117] "RemoveContainer" containerID="f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a" Jul 14 13:56:04.021652 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.021578 2564 scope.go:117] "RemoveContainer" containerID="f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a" Jul 14 13:56:04.021875 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:56:04.021853 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a\": container with ID starting with f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a not found: ID does not exist" containerID="f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a" Jul 14 13:56:04.021916 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.021879 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a"} err="failed to get container status \"f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a\": rpc error: code = NotFound desc = could not find container \"f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a\": container with ID starting with f75f442c30b0a5fa60a4c8682d04d2e637e6ec385640d83a33d12894053a4b8a not found: ID does not exist" Jul 14 13:56:04.033439 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.033410 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-565f9cc7f5-sns9v"] Jul 14 13:56:04.037215 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.037196 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-565f9cc7f5-sns9v"] Jul 14 13:56:04.907954 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:04.907928 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gz22v" Jul 14 13:56:05.369389 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:05.369361 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d9e7588-5f55-49fe-9ac6-e59d234c2235" path="/var/lib/kubelet/pods/6d9e7588-5f55-49fe-9ac6-e59d234c2235/volumes" Jul 14 13:56:07.141777 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:07.141752 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:56:07.145364 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:07.145346 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-595cf47db7-fhfld" Jul 14 13:56:46.860804 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.860766 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw"] Jul 14 13:56:46.861250 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.861044 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d9e7588-5f55-49fe-9ac6-e59d234c2235" containerName="console" Jul 14 13:56:46.861250 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.861054 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d9e7588-5f55-49fe-9ac6-e59d234c2235" containerName="console" Jul 14 13:56:46.861250 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.861101 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d9e7588-5f55-49fe-9ac6-e59d234c2235" containerName="console" Jul 14 13:56:46.862952 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.862936 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:46.868374 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.868349 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 13:56:46.868506 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.868486 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9bqwm\"" Jul 14 13:56:46.868506 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.868486 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 13:56:46.868625 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.868602 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 13:56:46.868840 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.868827 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 13:56:46.869530 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.869514 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 13:56:46.873566 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.873547 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 13:56:46.883101 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.883077 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw"] Jul 14 13:56:46.956152 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.956124 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-telemeter-client-tls\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:46.956152 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.956154 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-federate-client-tls\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:46.956405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.956182 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-serving-certs-ca-bundle\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:46.956405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.956227 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:46.956405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.956327 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-metrics-client-ca\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:46.956405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.956369 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twng8\" (UniqueName: \"kubernetes.io/projected/b331fc21-d7c9-45b9-811e-cdd1338cf182-kube-api-access-twng8\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:46.956405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.956387 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-secret-telemeter-client\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:46.956562 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:46.956452 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.056995 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.056945 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-telemeter-client-tls\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.056995 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.056995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-federate-client-tls\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.057203 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.057116 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-serving-certs-ca-bundle\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.057203 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.057156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.057272 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.057196 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-metrics-client-ca\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.057272 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.057262 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twng8\" (UniqueName: \"kubernetes.io/projected/b331fc21-d7c9-45b9-811e-cdd1338cf182-kube-api-access-twng8\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.057394 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.057318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-secret-telemeter-client\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.057394 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.057362 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.057973 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.057945 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-serving-certs-ca-bundle\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.058147 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.058123 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-telemeter-trusted-ca-bundle\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.058270 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.057978 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b331fc21-d7c9-45b9-811e-cdd1338cf182-metrics-client-ca\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.059726 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.059699 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-federate-client-tls\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.059819 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.059706 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-telemeter-client-tls\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.060030 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.060012 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.060067 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.060017 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b331fc21-d7c9-45b9-811e-cdd1338cf182-secret-telemeter-client\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.067494 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.067474 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twng8\" (UniqueName: \"kubernetes.io/projected/b331fc21-d7c9-45b9-811e-cdd1338cf182-kube-api-access-twng8\") pod \"telemeter-client-85c45dcbd6-rl5tw\" (UID: \"b331fc21-d7c9-45b9-811e-cdd1338cf182\") " pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.172338 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.172225 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" Jul 14 13:56:47.306768 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:47.306739 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw"] Jul 14 13:56:47.308954 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:56:47.308924 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb331fc21_d7c9_45b9_811e_cdd1338cf182.slice/crio-0327fe8f9fd4ecc5ba6fd83885c3829ef68c0468da1f68fb0c9d8c0d7bcbb2ad WatchSource:0}: Error finding container 0327fe8f9fd4ecc5ba6fd83885c3829ef68c0468da1f68fb0c9d8c0d7bcbb2ad: Status 404 returned error can't find the container with id 0327fe8f9fd4ecc5ba6fd83885c3829ef68c0468da1f68fb0c9d8c0d7bcbb2ad Jul 14 13:56:48.171967 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:48.171922 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" event={"ID":"b331fc21-d7c9-45b9-811e-cdd1338cf182","Type":"ContainerStarted","Data":"0327fe8f9fd4ecc5ba6fd83885c3829ef68c0468da1f68fb0c9d8c0d7bcbb2ad"} Jul 14 13:56:50.182389 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:50.182353 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" event={"ID":"b331fc21-d7c9-45b9-811e-cdd1338cf182","Type":"ContainerStarted","Data":"5969bf700d769b43df94f88a8438b8e336fb882b7d3f2b7066887e51252512ab"} Jul 14 13:56:50.182779 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:50.182422 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" event={"ID":"b331fc21-d7c9-45b9-811e-cdd1338cf182","Type":"ContainerStarted","Data":"5a399bdda591a11c1a1efc61dd176b56ab10661895ded6a77a34d2855e6fecbf"} Jul 14 13:56:50.182779 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:50.182448 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" event={"ID":"b331fc21-d7c9-45b9-811e-cdd1338cf182","Type":"ContainerStarted","Data":"a7b36459835afccce1e4734722b3785fbe8caf16c00efbd357750f041774a1d3"} Jul 14 13:56:50.208143 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:50.208066 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-85c45dcbd6-rl5tw" podStartSLOduration=2.396935416 podStartE2EDuration="4.208048143s" totalImagesPullingTime="1.811112727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:56:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:56:50.205960771 +0000 UTC m=+149.432643692" watchObservedRunningTime="2026-07-14 13:56:50.208048143 +0000 UTC m=+149.434731059" Jul 14 13:56:51.005217 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.005184 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b96b8f6fb-2cmtn"] Jul 14 13:56:51.007482 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.007463 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.018390 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.018328 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b96b8f6fb-2cmtn"] Jul 14 13:56:51.090544 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.090506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-service-ca\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.090740 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.090559 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-config\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.090740 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.090583 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-oauth-config\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.090740 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.090674 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-serving-cert\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.090740 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.090706 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtjsn\" (UniqueName: \"kubernetes.io/projected/e47a62a7-0acb-42e9-b22c-4658831e5d22-kube-api-access-wtjsn\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.090919 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.090745 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-oauth-serving-cert\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.090919 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.090794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-trusted-ca-bundle\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.192879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.192049 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-service-ca\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.192879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.192124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-config\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.192879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.192154 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-oauth-config\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.192879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.192195 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-serving-cert\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.192879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.192217 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtjsn\" (UniqueName: \"kubernetes.io/projected/e47a62a7-0acb-42e9-b22c-4658831e5d22-kube-api-access-wtjsn\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.192879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.192249 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-oauth-serving-cert\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.192879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.192299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-trusted-ca-bundle\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.193672 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.193426 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-trusted-ca-bundle\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.193672 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.193603 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-service-ca\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.193672 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.193628 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-config\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.193803 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.193726 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-oauth-serving-cert\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.195730 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.195704 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-oauth-config\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.197832 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.197805 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-serving-cert\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.201104 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.201081 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtjsn\" (UniqueName: \"kubernetes.io/projected/e47a62a7-0acb-42e9-b22c-4658831e5d22-kube-api-access-wtjsn\") pod \"console-6b96b8f6fb-2cmtn\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.316575 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.316536 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:56:51.446171 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:51.446149 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b96b8f6fb-2cmtn"] Jul 14 13:56:51.448683 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:56:51.448649 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode47a62a7_0acb_42e9_b22c_4658831e5d22.slice/crio-05c995c7c5281d6c8144a49e5b6084cf81a5f630a9f42fc2260075a5e2581217 WatchSource:0}: Error finding container 05c995c7c5281d6c8144a49e5b6084cf81a5f630a9f42fc2260075a5e2581217: Status 404 returned error can't find the container with id 05c995c7c5281d6c8144a49e5b6084cf81a5f630a9f42fc2260075a5e2581217 Jul 14 13:56:52.191491 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:52.191447 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b96b8f6fb-2cmtn" event={"ID":"e47a62a7-0acb-42e9-b22c-4658831e5d22","Type":"ContainerStarted","Data":"a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed"} Jul 14 13:56:52.191639 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:52.191537 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b96b8f6fb-2cmtn" event={"ID":"e47a62a7-0acb-42e9-b22c-4658831e5d22","Type":"ContainerStarted","Data":"05c995c7c5281d6c8144a49e5b6084cf81a5f630a9f42fc2260075a5e2581217"} Jul 14 13:56:52.210129 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:56:52.210085 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b96b8f6fb-2cmtn" podStartSLOduration=2.210071077 podStartE2EDuration="2.210071077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:56:52.209624344 +0000 UTC m=+151.436307272" watchObservedRunningTime="2026-07-14 13:56:52.210071077 +0000 UTC m=+151.436753985" Jul 14 13:57:01.316805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:01.316768 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:57:01.317371 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:01.316836 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:57:01.321741 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:01.321720 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:57:02.229976 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.229946 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:57:02.276807 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.276772 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79ccdbc5dc-gs5bm"] Jul 14 13:57:02.347773 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.347741 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f76975456-27tr6"] Jul 14 13:57:02.350290 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.350263 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.361023 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.361002 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f76975456-27tr6"] Jul 14 13:57:02.488036 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.487944 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-oauth-config\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.488036 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.487985 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px4x8\" (UniqueName: \"kubernetes.io/projected/d4c86535-833f-40de-ad9b-c9ab2e210162-kube-api-access-px4x8\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.488219 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.488062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-trusted-ca-bundle\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.488219 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.488103 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-oauth-serving-cert\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.488219 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.488165 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-service-ca\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.488219 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.488191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-console-config\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.488219 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.488209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-serving-cert\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.588919 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.588875 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-service-ca\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.588919 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.588920 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-console-config\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.588944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-serving-cert\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.588979 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-oauth-config\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.588994 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px4x8\" (UniqueName: \"kubernetes.io/projected/d4c86535-833f-40de-ad9b-c9ab2e210162-kube-api-access-px4x8\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.589016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-trusted-ca-bundle\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.589032 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-oauth-serving-cert\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589786 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.589761 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-oauth-serving-cert\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589910 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.589793 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-console-config\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589910 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.589795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-service-ca\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.589910 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.589875 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-trusted-ca-bundle\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.591486 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.591462 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-oauth-config\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.591579 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.591561 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-serving-cert\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.597456 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.597437 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px4x8\" (UniqueName: \"kubernetes.io/projected/d4c86535-833f-40de-ad9b-c9ab2e210162-kube-api-access-px4x8\") pod \"console-7f76975456-27tr6\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.659566 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.659534 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:02.787805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:02.787770 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f76975456-27tr6"] Jul 14 13:57:02.790808 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:57:02.790775 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c86535_833f_40de_ad9b_c9ab2e210162.slice/crio-fb2785a6358034713559e92e0718da8f43398867c3043aa96b121dc32cee521a WatchSource:0}: Error finding container fb2785a6358034713559e92e0718da8f43398867c3043aa96b121dc32cee521a: Status 404 returned error can't find the container with id fb2785a6358034713559e92e0718da8f43398867c3043aa96b121dc32cee521a Jul 14 13:57:03.230865 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:03.230831 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f76975456-27tr6" event={"ID":"d4c86535-833f-40de-ad9b-c9ab2e210162","Type":"ContainerStarted","Data":"b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174"} Jul 14 13:57:03.231046 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:03.230885 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f76975456-27tr6" event={"ID":"d4c86535-833f-40de-ad9b-c9ab2e210162","Type":"ContainerStarted","Data":"fb2785a6358034713559e92e0718da8f43398867c3043aa96b121dc32cee521a"} Jul 14 13:57:03.252199 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:03.252127 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f76975456-27tr6" podStartSLOduration=1.2521098959999999 podStartE2EDuration="1.252109896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:57:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:57:03.251331265 +0000 UTC m=+162.478014182" watchObservedRunningTime="2026-07-14 13:57:03.252109896 +0000 UTC m=+162.478792901" Jul 14 13:57:12.660450 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:12.660425 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:12.660870 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:12.660492 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:12.664914 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:12.664891 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:13.271251 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:13.271224 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f76975456-27tr6" Jul 14 13:57:13.318564 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:13.318532 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b96b8f6fb-2cmtn"] Jul 14 13:57:27.296698 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.296652 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79ccdbc5dc-gs5bm" podUID="cc0a2e29-e340-44a1-a425-f08fb8315616" containerName="console" containerID="cri-o://5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef" gracePeriod=15 Jul 14 13:57:27.536459 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.536437 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:57:27.570144 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570111 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-trusted-ca-bundle\") pod \"cc0a2e29-e340-44a1-a425-f08fb8315616\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " Jul 14 13:57:27.570318 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570184 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-oauth-serving-cert\") pod \"cc0a2e29-e340-44a1-a425-f08fb8315616\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " Jul 14 13:57:27.570318 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570270 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-service-ca\") pod \"cc0a2e29-e340-44a1-a425-f08fb8315616\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " Jul 14 13:57:27.570415 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570329 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8thvp\" (UniqueName: \"kubernetes.io/projected/cc0a2e29-e340-44a1-a425-f08fb8315616-kube-api-access-8thvp\") pod \"cc0a2e29-e340-44a1-a425-f08fb8315616\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " Jul 14 13:57:27.570415 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570351 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-serving-cert\") pod \"cc0a2e29-e340-44a1-a425-f08fb8315616\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " Jul 14 13:57:27.570415 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570401 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-console-config\") pod \"cc0a2e29-e340-44a1-a425-f08fb8315616\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " Jul 14 13:57:27.570561 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570431 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-oauth-config\") pod \"cc0a2e29-e340-44a1-a425-f08fb8315616\" (UID: \"cc0a2e29-e340-44a1-a425-f08fb8315616\") " Jul 14 13:57:27.570616 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570596 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cc0a2e29-e340-44a1-a425-f08fb8315616" (UID: "cc0a2e29-e340-44a1-a425-f08fb8315616"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:27.570674 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570541 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cc0a2e29-e340-44a1-a425-f08fb8315616" (UID: "cc0a2e29-e340-44a1-a425-f08fb8315616"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:27.570728 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570670 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-oauth-serving-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:27.570789 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570733 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-service-ca" (OuterVolumeSpecName: "service-ca") pod "cc0a2e29-e340-44a1-a425-f08fb8315616" (UID: "cc0a2e29-e340-44a1-a425-f08fb8315616"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:27.570878 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.570855 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-console-config" (OuterVolumeSpecName: "console-config") pod "cc0a2e29-e340-44a1-a425-f08fb8315616" (UID: "cc0a2e29-e340-44a1-a425-f08fb8315616"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:27.572531 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.572508 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cc0a2e29-e340-44a1-a425-f08fb8315616" (UID: "cc0a2e29-e340-44a1-a425-f08fb8315616"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:57:27.572630 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.572590 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0a2e29-e340-44a1-a425-f08fb8315616-kube-api-access-8thvp" (OuterVolumeSpecName: "kube-api-access-8thvp") pod "cc0a2e29-e340-44a1-a425-f08fb8315616" (UID: "cc0a2e29-e340-44a1-a425-f08fb8315616"). InnerVolumeSpecName "kube-api-access-8thvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:57:27.572630 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.572606 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cc0a2e29-e340-44a1-a425-f08fb8315616" (UID: "cc0a2e29-e340-44a1-a425-f08fb8315616"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:57:27.671303 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.671243 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-trusted-ca-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:27.671303 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.671272 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:27.671303 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.671302 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8thvp\" (UniqueName: \"kubernetes.io/projected/cc0a2e29-e340-44a1-a425-f08fb8315616-kube-api-access-8thvp\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:27.671303 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.671311 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-serving-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:27.671550 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.671321 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc0a2e29-e340-44a1-a425-f08fb8315616-console-config\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:27.671550 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:27.671330 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc0a2e29-e340-44a1-a425-f08fb8315616-console-oauth-config\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:28.327492 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.327455 2564 generic.go:356] "Generic (PLEG): container finished" podID="cc0a2e29-e340-44a1-a425-f08fb8315616" containerID="5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef" exitCode=0 Jul 14 13:57:28.327924 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.327504 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79ccdbc5dc-gs5bm" Jul 14 13:57:28.327924 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.327533 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79ccdbc5dc-gs5bm" event={"ID":"cc0a2e29-e340-44a1-a425-f08fb8315616","Type":"ContainerDied","Data":"5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef"} Jul 14 13:57:28.327924 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.327587 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79ccdbc5dc-gs5bm" event={"ID":"cc0a2e29-e340-44a1-a425-f08fb8315616","Type":"ContainerDied","Data":"2f49210dea0099561634cef482f5347ba16037b9b180c4167e864aebc6a04775"} Jul 14 13:57:28.327924 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.327606 2564 scope.go:117] "RemoveContainer" containerID="5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef" Jul 14 13:57:28.336299 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.336270 2564 scope.go:117] "RemoveContainer" containerID="5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef" Jul 14 13:57:28.336557 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:57:28.336537 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef\": container with ID starting with 5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef not found: ID does not exist" containerID="5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef" Jul 14 13:57:28.336603 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.336567 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef"} err="failed to get container status \"5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef\": rpc error: code = NotFound desc = could not find container \"5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef\": container with ID starting with 5fc8dfa112343fedd6f8e8ee4687e2fee617670fc31007dc31c10f77e57fd4ef not found: ID does not exist" Jul 14 13:57:28.348856 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.348837 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79ccdbc5dc-gs5bm"] Jul 14 13:57:28.351081 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:28.351061 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79ccdbc5dc-gs5bm"] Jul 14 13:57:29.368501 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:29.368469 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0a2e29-e340-44a1-a425-f08fb8315616" path="/var/lib/kubelet/pods/cc0a2e29-e340-44a1-a425-f08fb8315616/volumes" Jul 14 13:57:38.339346 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.339299 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b96b8f6fb-2cmtn" podUID="e47a62a7-0acb-42e9-b22c-4658831e5d22" containerName="console" containerID="cri-o://a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed" gracePeriod=15 Jul 14 13:57:38.577928 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.577903 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:57:38.654464 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654381 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-serving-cert\") pod \"e47a62a7-0acb-42e9-b22c-4658831e5d22\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " Jul 14 13:57:38.654464 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654422 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-service-ca\") pod \"e47a62a7-0acb-42e9-b22c-4658831e5d22\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " Jul 14 13:57:38.654464 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654439 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-oauth-config\") pod \"e47a62a7-0acb-42e9-b22c-4658831e5d22\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " Jul 14 13:57:38.654695 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654487 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-config\") pod \"e47a62a7-0acb-42e9-b22c-4658831e5d22\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " Jul 14 13:57:38.654695 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654512 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-oauth-serving-cert\") pod \"e47a62a7-0acb-42e9-b22c-4658831e5d22\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " Jul 14 13:57:38.654695 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654562 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-trusted-ca-bundle\") pod \"e47a62a7-0acb-42e9-b22c-4658831e5d22\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " Jul 14 13:57:38.654695 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654603 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtjsn\" (UniqueName: \"kubernetes.io/projected/e47a62a7-0acb-42e9-b22c-4658831e5d22-kube-api-access-wtjsn\") pod \"e47a62a7-0acb-42e9-b22c-4658831e5d22\" (UID: \"e47a62a7-0acb-42e9-b22c-4658831e5d22\") " Jul 14 13:57:38.655012 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654955 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-service-ca" (OuterVolumeSpecName: "service-ca") pod "e47a62a7-0acb-42e9-b22c-4658831e5d22" (UID: "e47a62a7-0acb-42e9-b22c-4658831e5d22"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:38.655141 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.654952 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e47a62a7-0acb-42e9-b22c-4658831e5d22" (UID: "e47a62a7-0acb-42e9-b22c-4658831e5d22"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:38.655141 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.655090 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e47a62a7-0acb-42e9-b22c-4658831e5d22" (UID: "e47a62a7-0acb-42e9-b22c-4658831e5d22"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:38.655253 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.655144 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-config" (OuterVolumeSpecName: "console-config") pod "e47a62a7-0acb-42e9-b22c-4658831e5d22" (UID: "e47a62a7-0acb-42e9-b22c-4658831e5d22"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 13:57:38.656672 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.656649 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e47a62a7-0acb-42e9-b22c-4658831e5d22" (UID: "e47a62a7-0acb-42e9-b22c-4658831e5d22"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:57:38.656758 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.656668 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e47a62a7-0acb-42e9-b22c-4658831e5d22" (UID: "e47a62a7-0acb-42e9-b22c-4658831e5d22"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 13:57:38.656758 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.656674 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e47a62a7-0acb-42e9-b22c-4658831e5d22-kube-api-access-wtjsn" (OuterVolumeSpecName: "kube-api-access-wtjsn") pod "e47a62a7-0acb-42e9-b22c-4658831e5d22" (UID: "e47a62a7-0acb-42e9-b22c-4658831e5d22"). InnerVolumeSpecName "kube-api-access-wtjsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:57:38.755514 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.755482 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-config\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:38.755514 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.755511 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-oauth-serving-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:38.755514 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.755522 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-trusted-ca-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:38.755746 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.755530 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtjsn\" (UniqueName: \"kubernetes.io/projected/e47a62a7-0acb-42e9-b22c-4658831e5d22-kube-api-access-wtjsn\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:38.755746 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.755538 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-serving-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:38.755746 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.755547 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e47a62a7-0acb-42e9-b22c-4658831e5d22-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:38.755746 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:38.755556 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e47a62a7-0acb-42e9-b22c-4658831e5d22-console-oauth-config\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:57:39.367833 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.367795 2564 generic.go:356] "Generic (PLEG): container finished" podID="e47a62a7-0acb-42e9-b22c-4658831e5d22" containerID="a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed" exitCode=0 Jul 14 13:57:39.368258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.367908 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b96b8f6fb-2cmtn" Jul 14 13:57:39.368701 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.368680 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b96b8f6fb-2cmtn" event={"ID":"e47a62a7-0acb-42e9-b22c-4658831e5d22","Type":"ContainerDied","Data":"a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed"} Jul 14 13:57:39.368741 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.368717 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b96b8f6fb-2cmtn" event={"ID":"e47a62a7-0acb-42e9-b22c-4658831e5d22","Type":"ContainerDied","Data":"05c995c7c5281d6c8144a49e5b6084cf81a5f630a9f42fc2260075a5e2581217"} Jul 14 13:57:39.368773 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.368743 2564 scope.go:117] "RemoveContainer" containerID="a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed" Jul 14 13:57:39.377089 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.376936 2564 scope.go:117] "RemoveContainer" containerID="a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed" Jul 14 13:57:39.377226 ip-10-0-142-141 kubenswrapper[2564]: E0714 13:57:39.377205 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed\": container with ID starting with a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed not found: ID does not exist" containerID="a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed" Jul 14 13:57:39.377309 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.377237 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed"} err="failed to get container status \"a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed\": rpc error: code = NotFound desc = could not find container \"a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed\": container with ID starting with a71d6e90f075aa3d65d0aa00b2cd641cab4f25eb8556c8ddac6cc941a86debed not found: ID does not exist" Jul 14 13:57:39.392846 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.392824 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b96b8f6fb-2cmtn"] Jul 14 13:57:39.396195 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:39.396168 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b96b8f6fb-2cmtn"] Jul 14 13:57:41.369204 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:57:41.369171 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e47a62a7-0acb-42e9-b22c-4658831e5d22" path="/var/lib/kubelet/pods/e47a62a7-0acb-42e9-b22c-4658831e5d22/volumes" Jul 14 13:58:20.727407 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.727377 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7"] Jul 14 13:58:20.727843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.727670 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0a2e29-e340-44a1-a425-f08fb8315616" containerName="console" Jul 14 13:58:20.727843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.727682 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0a2e29-e340-44a1-a425-f08fb8315616" containerName="console" Jul 14 13:58:20.727843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.727697 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e47a62a7-0acb-42e9-b22c-4658831e5d22" containerName="console" Jul 14 13:58:20.727843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.727702 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e47a62a7-0acb-42e9-b22c-4658831e5d22" containerName="console" Jul 14 13:58:20.727843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.727747 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0a2e29-e340-44a1-a425-f08fb8315616" containerName="console" Jul 14 13:58:20.727843 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.727756 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e47a62a7-0acb-42e9-b22c-4658831e5d22" containerName="console" Jul 14 13:58:20.730642 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.730621 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:20.733193 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.733168 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 13:58:20.733351 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.733218 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4scxk\"" Jul 14 13:58:20.733351 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.733244 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 13:58:20.740133 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.740104 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7"] Jul 14 13:58:20.893076 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.893039 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:20.893301 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.893214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:20.893301 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.893258 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x962\" (UniqueName: \"kubernetes.io/projected/eae07476-28bd-4793-8b4b-a7be7026989d-kube-api-access-7x962\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:20.993770 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.993671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:20.993770 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.993720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x962\" (UniqueName: \"kubernetes.io/projected/eae07476-28bd-4793-8b4b-a7be7026989d-kube-api-access-7x962\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:20.993770 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.993745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:20.994073 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.994055 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:20.994150 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:20.994126 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:21.002901 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:21.002871 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x962\" (UniqueName: \"kubernetes.io/projected/eae07476-28bd-4793-8b4b-a7be7026989d-kube-api-access-7x962\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:21.039763 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:21.039723 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:21.168090 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:21.168060 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7"] Jul 14 13:58:21.171157 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:58:21.171130 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeae07476_28bd_4793_8b4b_a7be7026989d.slice/crio-1e5a11e0c19020052cf86b390dad727f68aec4a99feb4c553801bf30157b9c0a WatchSource:0}: Error finding container 1e5a11e0c19020052cf86b390dad727f68aec4a99feb4c553801bf30157b9c0a: Status 404 returned error can't find the container with id 1e5a11e0c19020052cf86b390dad727f68aec4a99feb4c553801bf30157b9c0a Jul 14 13:58:21.520343 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:21.520306 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" event={"ID":"eae07476-28bd-4793-8b4b-a7be7026989d","Type":"ContainerStarted","Data":"1e5a11e0c19020052cf86b390dad727f68aec4a99feb4c553801bf30157b9c0a"} Jul 14 13:58:30.559345 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:30.559312 2564 generic.go:356] "Generic (PLEG): container finished" podID="eae07476-28bd-4793-8b4b-a7be7026989d" containerID="556d4a275f98b15ba5818ef7907bbcb50e4682ce65728aec81a3ebbd5161970c" exitCode=0 Jul 14 13:58:30.559747 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:30.559364 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" event={"ID":"eae07476-28bd-4793-8b4b-a7be7026989d","Type":"ContainerDied","Data":"556d4a275f98b15ba5818ef7907bbcb50e4682ce65728aec81a3ebbd5161970c"} Jul 14 13:58:33.573302 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:33.573253 2564 generic.go:356] "Generic (PLEG): container finished" podID="eae07476-28bd-4793-8b4b-a7be7026989d" containerID="04ddfac8f2aad7ee78b7eaa3c12a434db2eba38e2de19c499c7454ff3449f23c" exitCode=0 Jul 14 13:58:33.573671 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:33.573341 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" event={"ID":"eae07476-28bd-4793-8b4b-a7be7026989d","Type":"ContainerDied","Data":"04ddfac8f2aad7ee78b7eaa3c12a434db2eba38e2de19c499c7454ff3449f23c"} Jul 14 13:58:42.612132 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:42.612099 2564 generic.go:356] "Generic (PLEG): container finished" podID="eae07476-28bd-4793-8b4b-a7be7026989d" containerID="afbad2424214d14fe537e5af72b5a1255e6e56c0676c9db6cbc7f516d4435777" exitCode=0 Jul 14 13:58:42.612536 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:42.612138 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" event={"ID":"eae07476-28bd-4793-8b4b-a7be7026989d","Type":"ContainerDied","Data":"afbad2424214d14fe537e5af72b5a1255e6e56c0676c9db6cbc7f516d4435777"} Jul 14 13:58:43.774903 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.774870 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:43.896101 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.896009 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-bundle\") pod \"eae07476-28bd-4793-8b4b-a7be7026989d\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " Jul 14 13:58:43.896101 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.896050 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-util\") pod \"eae07476-28bd-4793-8b4b-a7be7026989d\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " Jul 14 13:58:43.896101 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.896072 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x962\" (UniqueName: \"kubernetes.io/projected/eae07476-28bd-4793-8b4b-a7be7026989d-kube-api-access-7x962\") pod \"eae07476-28bd-4793-8b4b-a7be7026989d\" (UID: \"eae07476-28bd-4793-8b4b-a7be7026989d\") " Jul 14 13:58:43.896871 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.896841 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-bundle" (OuterVolumeSpecName: "bundle") pod "eae07476-28bd-4793-8b4b-a7be7026989d" (UID: "eae07476-28bd-4793-8b4b-a7be7026989d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:58:43.898789 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.898761 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eae07476-28bd-4793-8b4b-a7be7026989d-kube-api-access-7x962" (OuterVolumeSpecName: "kube-api-access-7x962") pod "eae07476-28bd-4793-8b4b-a7be7026989d" (UID: "eae07476-28bd-4793-8b4b-a7be7026989d"). InnerVolumeSpecName "kube-api-access-7x962". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:58:43.901358 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.901329 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-util" (OuterVolumeSpecName: "util") pod "eae07476-28bd-4793-8b4b-a7be7026989d" (UID: "eae07476-28bd-4793-8b4b-a7be7026989d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:58:43.997023 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.996984 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-util\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:58:43.997023 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.997015 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x962\" (UniqueName: \"kubernetes.io/projected/eae07476-28bd-4793-8b4b-a7be7026989d-kube-api-access-7x962\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:58:43.997023 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:43.997026 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eae07476-28bd-4793-8b4b-a7be7026989d-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:58:44.621376 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:44.621345 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" event={"ID":"eae07476-28bd-4793-8b4b-a7be7026989d","Type":"ContainerDied","Data":"1e5a11e0c19020052cf86b390dad727f68aec4a99feb4c553801bf30157b9c0a"} Jul 14 13:58:44.621570 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:44.621391 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e5a11e0c19020052cf86b390dad727f68aec4a99feb4c553801bf30157b9c0a" Jul 14 13:58:44.621570 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:44.621428 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gskz7" Jul 14 13:58:48.778267 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.778230 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2"] Jul 14 13:58:48.778725 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.778542 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eae07476-28bd-4793-8b4b-a7be7026989d" containerName="extract" Jul 14 13:58:48.778725 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.778553 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae07476-28bd-4793-8b4b-a7be7026989d" containerName="extract" Jul 14 13:58:48.778725 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.778566 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eae07476-28bd-4793-8b4b-a7be7026989d" containerName="pull" Jul 14 13:58:48.778725 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.778571 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae07476-28bd-4793-8b4b-a7be7026989d" containerName="pull" Jul 14 13:58:48.778725 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.778579 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eae07476-28bd-4793-8b4b-a7be7026989d" containerName="util" Jul 14 13:58:48.778725 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.778584 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eae07476-28bd-4793-8b4b-a7be7026989d" containerName="util" Jul 14 13:58:48.778725 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.778633 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eae07476-28bd-4793-8b4b-a7be7026989d" containerName="extract" Jul 14 13:58:48.780523 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.780509 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" Jul 14 13:58:48.783300 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.783256 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:58:48.783300 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.783270 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-bh64z\"" Jul 14 13:58:48.783570 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.783270 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 14 13:58:48.793969 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.793943 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2"] Jul 14 13:58:48.940784 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.940737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c0dfa7e-5515-421d-b7dc-b54204e876e3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d86p2\" (UID: \"5c0dfa7e-5515-421d-b7dc-b54204e876e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" Jul 14 13:58:48.940784 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:48.940781 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zwgz\" (UniqueName: \"kubernetes.io/projected/5c0dfa7e-5515-421d-b7dc-b54204e876e3-kube-api-access-5zwgz\") pod \"cert-manager-operator-controller-manager-55c444bf86-d86p2\" (UID: \"5c0dfa7e-5515-421d-b7dc-b54204e876e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" Jul 14 13:58:49.042110 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:49.042029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c0dfa7e-5515-421d-b7dc-b54204e876e3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d86p2\" (UID: \"5c0dfa7e-5515-421d-b7dc-b54204e876e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" Jul 14 13:58:49.042110 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:49.042075 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zwgz\" (UniqueName: \"kubernetes.io/projected/5c0dfa7e-5515-421d-b7dc-b54204e876e3-kube-api-access-5zwgz\") pod \"cert-manager-operator-controller-manager-55c444bf86-d86p2\" (UID: \"5c0dfa7e-5515-421d-b7dc-b54204e876e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" Jul 14 13:58:49.042458 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:49.042440 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c0dfa7e-5515-421d-b7dc-b54204e876e3-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d86p2\" (UID: \"5c0dfa7e-5515-421d-b7dc-b54204e876e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" Jul 14 13:58:49.052345 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:49.052321 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zwgz\" (UniqueName: \"kubernetes.io/projected/5c0dfa7e-5515-421d-b7dc-b54204e876e3-kube-api-access-5zwgz\") pod \"cert-manager-operator-controller-manager-55c444bf86-d86p2\" (UID: \"5c0dfa7e-5515-421d-b7dc-b54204e876e3\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" Jul 14 13:58:49.090181 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:49.090146 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" Jul 14 13:58:49.220081 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:49.220050 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2"] Jul 14 13:58:49.223690 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:58:49.223667 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0dfa7e_5515_421d_b7dc_b54204e876e3.slice/crio-aa5a6fee0afd7403a69b1df890078fbeaed177114e7c3c37cb3451aa4f7a9fea WatchSource:0}: Error finding container aa5a6fee0afd7403a69b1df890078fbeaed177114e7c3c37cb3451aa4f7a9fea: Status 404 returned error can't find the container with id aa5a6fee0afd7403a69b1df890078fbeaed177114e7c3c37cb3451aa4f7a9fea Jul 14 13:58:49.645505 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:49.645466 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" event={"ID":"5c0dfa7e-5515-421d-b7dc-b54204e876e3","Type":"ContainerStarted","Data":"aa5a6fee0afd7403a69b1df890078fbeaed177114e7c3c37cb3451aa4f7a9fea"} Jul 14 13:58:51.656694 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:51.656661 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" event={"ID":"5c0dfa7e-5515-421d-b7dc-b54204e876e3","Type":"ContainerStarted","Data":"ee4a3dd78300e46d20705ec2cb1074792966b3243abfae06de4749983ff4f0ab"} Jul 14 13:58:51.681752 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:51.681696 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d86p2" podStartSLOduration=1.909895326 podStartE2EDuration="3.681677014s" totalImagesPullingTime="1.771781688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:58:51.678509665 +0000 UTC m=+270.905192611" watchObservedRunningTime="2026-07-14 13:58:51.681677014 +0000 UTC m=+270.908359931" Jul 14 13:58:52.606931 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.606895 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn"] Jul 14 13:58:52.609582 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.609565 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.612365 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.612347 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 13:58:52.612471 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.612396 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 13:58:52.613646 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.613629 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4scxk\"" Jul 14 13:58:52.618980 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.618957 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn"] Jul 14 13:58:52.773157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.773122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.773157 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.773160 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndrp6\" (UniqueName: \"kubernetes.io/projected/36f9cf40-5020-406c-9976-e63e394014b3-kube-api-access-ndrp6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.773632 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.773262 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.874351 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.874250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.874483 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.874419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.874483 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.874472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndrp6\" (UniqueName: \"kubernetes.io/projected/36f9cf40-5020-406c-9976-e63e394014b3-kube-api-access-ndrp6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.874696 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.874674 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.874739 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.874681 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.885610 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.885579 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndrp6\" (UniqueName: \"kubernetes.io/projected/36f9cf40-5020-406c-9976-e63e394014b3-kube-api-access-ndrp6\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:52.919571 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:52.919534 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:53.031434 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.031405 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dscnr"] Jul 14 13:58:53.033530 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.033515 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:53.036121 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.036092 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 13:58:53.036260 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.036120 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 13:58:53.036260 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.036147 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-54dcb\"" Jul 14 13:58:53.045966 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.045943 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dscnr"] Jul 14 13:58:53.050406 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.050365 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn"] Jul 14 13:58:53.052637 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:58:53.052614 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f9cf40_5020_406c_9976_e63e394014b3.slice/crio-51602b959ad4ed8861f602b654e8d7e2f107c06c1b7a0dab0d7fcb9315e14285 WatchSource:0}: Error finding container 51602b959ad4ed8861f602b654e8d7e2f107c06c1b7a0dab0d7fcb9315e14285: Status 404 returned error can't find the container with id 51602b959ad4ed8861f602b654e8d7e2f107c06c1b7a0dab0d7fcb9315e14285 Jul 14 13:58:53.178039 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.178007 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/234a10ad-8116-47a8-bc58-035d853f1936-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dscnr\" (UID: \"234a10ad-8116-47a8-bc58-035d853f1936\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:53.178206 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.178072 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gwp7\" (UniqueName: \"kubernetes.io/projected/234a10ad-8116-47a8-bc58-035d853f1936-kube-api-access-5gwp7\") pod \"cert-manager-webhook-6486ffd997-dscnr\" (UID: \"234a10ad-8116-47a8-bc58-035d853f1936\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:53.279430 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.279401 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/234a10ad-8116-47a8-bc58-035d853f1936-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dscnr\" (UID: \"234a10ad-8116-47a8-bc58-035d853f1936\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:53.279604 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.279457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gwp7\" (UniqueName: \"kubernetes.io/projected/234a10ad-8116-47a8-bc58-035d853f1936-kube-api-access-5gwp7\") pod \"cert-manager-webhook-6486ffd997-dscnr\" (UID: \"234a10ad-8116-47a8-bc58-035d853f1936\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:53.293689 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.293656 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gwp7\" (UniqueName: \"kubernetes.io/projected/234a10ad-8116-47a8-bc58-035d853f1936-kube-api-access-5gwp7\") pod \"cert-manager-webhook-6486ffd997-dscnr\" (UID: \"234a10ad-8116-47a8-bc58-035d853f1936\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:53.293811 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.293661 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/234a10ad-8116-47a8-bc58-035d853f1936-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-dscnr\" (UID: \"234a10ad-8116-47a8-bc58-035d853f1936\") " pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:53.351347 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.351313 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:53.480098 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.480069 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-dscnr"] Jul 14 13:58:53.482089 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:58:53.482056 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod234a10ad_8116_47a8_bc58_035d853f1936.slice/crio-0e29072c25917161288996a276f00c2e465ca391f36fbaf3b7bc0de63ba2cb94 WatchSource:0}: Error finding container 0e29072c25917161288996a276f00c2e465ca391f36fbaf3b7bc0de63ba2cb94: Status 404 returned error can't find the container with id 0e29072c25917161288996a276f00c2e465ca391f36fbaf3b7bc0de63ba2cb94 Jul 14 13:58:53.665938 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.665844 2564 generic.go:356] "Generic (PLEG): container finished" podID="36f9cf40-5020-406c-9976-e63e394014b3" containerID="ca7429c9994c58c6673e22368e9770898cacf314f3538e11c98856225a1cc57c" exitCode=0 Jul 14 13:58:53.665938 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.665931 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" event={"ID":"36f9cf40-5020-406c-9976-e63e394014b3","Type":"ContainerDied","Data":"ca7429c9994c58c6673e22368e9770898cacf314f3538e11c98856225a1cc57c"} Jul 14 13:58:53.666160 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.665978 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" event={"ID":"36f9cf40-5020-406c-9976-e63e394014b3","Type":"ContainerStarted","Data":"51602b959ad4ed8861f602b654e8d7e2f107c06c1b7a0dab0d7fcb9315e14285"} Jul 14 13:58:53.667129 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:53.667107 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" event={"ID":"234a10ad-8116-47a8-bc58-035d853f1936","Type":"ContainerStarted","Data":"0e29072c25917161288996a276f00c2e465ca391f36fbaf3b7bc0de63ba2cb94"} Jul 14 13:58:55.319623 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.319586 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg"] Jul 14 13:58:55.325996 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.325967 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" Jul 14 13:58:55.328674 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.328648 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lh54z\"" Jul 14 13:58:55.333095 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.333064 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg"] Jul 14 13:58:55.498239 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.498192 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39a74309-48f3-4121-b056-c5413aa786cc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mxgmg\" (UID: \"39a74309-48f3-4121-b056-c5413aa786cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" Jul 14 13:58:55.498436 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.498264 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd7jk\" (UniqueName: \"kubernetes.io/projected/39a74309-48f3-4121-b056-c5413aa786cc-kube-api-access-hd7jk\") pod \"cert-manager-cainjector-559cdd8c87-mxgmg\" (UID: \"39a74309-48f3-4121-b056-c5413aa786cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" Jul 14 13:58:55.598853 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.598761 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39a74309-48f3-4121-b056-c5413aa786cc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mxgmg\" (UID: \"39a74309-48f3-4121-b056-c5413aa786cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" Jul 14 13:58:55.598853 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.598827 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd7jk\" (UniqueName: \"kubernetes.io/projected/39a74309-48f3-4121-b056-c5413aa786cc-kube-api-access-hd7jk\") pod \"cert-manager-cainjector-559cdd8c87-mxgmg\" (UID: \"39a74309-48f3-4121-b056-c5413aa786cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" Jul 14 13:58:55.607352 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.607325 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd7jk\" (UniqueName: \"kubernetes.io/projected/39a74309-48f3-4121-b056-c5413aa786cc-kube-api-access-hd7jk\") pod \"cert-manager-cainjector-559cdd8c87-mxgmg\" (UID: \"39a74309-48f3-4121-b056-c5413aa786cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" Jul 14 13:58:55.607495 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.607438 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39a74309-48f3-4121-b056-c5413aa786cc-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mxgmg\" (UID: \"39a74309-48f3-4121-b056-c5413aa786cc\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" Jul 14 13:58:55.638634 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:55.638591 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" Jul 14 13:58:56.877894 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:56.877870 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg"] Jul 14 13:58:56.881149 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:58:56.881121 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39a74309_48f3_4121_b056_c5413aa786cc.slice/crio-709744556c86de701b85b02cc10baa78adff104699b7d378f1413494da66b568 WatchSource:0}: Error finding container 709744556c86de701b85b02cc10baa78adff104699b7d378f1413494da66b568: Status 404 returned error can't find the container with id 709744556c86de701b85b02cc10baa78adff104699b7d378f1413494da66b568 Jul 14 13:58:57.686852 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:57.686814 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" event={"ID":"39a74309-48f3-4121-b056-c5413aa786cc","Type":"ContainerStarted","Data":"a395452056f1ecd0e08daacf2c484a99e3f187374388984ad4c1dac809eb9e52"} Jul 14 13:58:57.687041 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:57.686862 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" event={"ID":"39a74309-48f3-4121-b056-c5413aa786cc","Type":"ContainerStarted","Data":"709744556c86de701b85b02cc10baa78adff104699b7d378f1413494da66b568"} Jul 14 13:58:57.688586 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:57.688560 2564 generic.go:356] "Generic (PLEG): container finished" podID="36f9cf40-5020-406c-9976-e63e394014b3" containerID="b7a3dd51493bc89cda3fbb2aa96e662834047bf8bec70ed32f49274a3e944e1f" exitCode=0 Jul 14 13:58:57.688689 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:57.688607 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" event={"ID":"36f9cf40-5020-406c-9976-e63e394014b3","Type":"ContainerDied","Data":"b7a3dd51493bc89cda3fbb2aa96e662834047bf8bec70ed32f49274a3e944e1f"} Jul 14 13:58:57.690174 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:57.690115 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" event={"ID":"234a10ad-8116-47a8-bc58-035d853f1936","Type":"ContainerStarted","Data":"101d53baeceae9c31f5b3a6c0d3abac63ebf7e8f30a28163d1301c7f46c7b3d4"} Jul 14 13:58:57.690242 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:57.690231 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:58:57.704888 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:57.704842 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mxgmg" podStartSLOduration=2.704825032 podStartE2EDuration="2.704825032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:58:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:58:57.704121439 +0000 UTC m=+276.930804376" watchObservedRunningTime="2026-07-14 13:58:57.704825032 +0000 UTC m=+276.931507948" Jul 14 13:58:57.724273 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:57.724217 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" podStartSLOduration=1.4694405910000001 podStartE2EDuration="4.724200679s" totalImagesPullingTime="3.254760088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:58:57.724170887 +0000 UTC m=+276.950853823" watchObservedRunningTime="2026-07-14 13:58:57.724200679 +0000 UTC m=+276.950883596" Jul 14 13:58:58.696022 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:58.695990 2564 generic.go:356] "Generic (PLEG): container finished" podID="36f9cf40-5020-406c-9976-e63e394014b3" containerID="a1d135eb9e0b935226c939a639e82cd8b4dddfe9431c9d25637f7ad7e0fc32fd" exitCode=0 Jul 14 13:58:58.696407 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:58.696085 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" event={"ID":"36f9cf40-5020-406c-9976-e63e394014b3","Type":"ContainerDied","Data":"a1d135eb9e0b935226c939a639e82cd8b4dddfe9431c9d25637f7ad7e0fc32fd"} Jul 14 13:58:59.824429 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:59.824404 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:58:59.942827 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:59.942789 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-bundle\") pod \"36f9cf40-5020-406c-9976-e63e394014b3\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " Jul 14 13:58:59.942997 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:59.942837 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndrp6\" (UniqueName: \"kubernetes.io/projected/36f9cf40-5020-406c-9976-e63e394014b3-kube-api-access-ndrp6\") pod \"36f9cf40-5020-406c-9976-e63e394014b3\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " Jul 14 13:58:59.942997 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:59.942862 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-util\") pod \"36f9cf40-5020-406c-9976-e63e394014b3\" (UID: \"36f9cf40-5020-406c-9976-e63e394014b3\") " Jul 14 13:58:59.943209 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:59.943187 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-bundle" (OuterVolumeSpecName: "bundle") pod "36f9cf40-5020-406c-9976-e63e394014b3" (UID: "36f9cf40-5020-406c-9976-e63e394014b3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:58:59.944974 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:59.944944 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f9cf40-5020-406c-9976-e63e394014b3-kube-api-access-ndrp6" (OuterVolumeSpecName: "kube-api-access-ndrp6") pod "36f9cf40-5020-406c-9976-e63e394014b3" (UID: "36f9cf40-5020-406c-9976-e63e394014b3"). InnerVolumeSpecName "kube-api-access-ndrp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:58:59.947228 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:58:59.947202 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-util" (OuterVolumeSpecName: "util") pod "36f9cf40-5020-406c-9976-e63e394014b3" (UID: "36f9cf40-5020-406c-9976-e63e394014b3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:59:00.043949 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:00.043860 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:59:00.043949 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:00.043895 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndrp6\" (UniqueName: \"kubernetes.io/projected/36f9cf40-5020-406c-9976-e63e394014b3-kube-api-access-ndrp6\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:59:00.043949 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:00.043904 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f9cf40-5020-406c-9976-e63e394014b3-util\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:59:00.707480 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:00.707455 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" Jul 14 13:59:00.707616 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:00.707479 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhdnn" event={"ID":"36f9cf40-5020-406c-9976-e63e394014b3","Type":"ContainerDied","Data":"51602b959ad4ed8861f602b654e8d7e2f107c06c1b7a0dab0d7fcb9315e14285"} Jul 14 13:59:00.707616 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:00.707525 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51602b959ad4ed8861f602b654e8d7e2f107c06c1b7a0dab0d7fcb9315e14285" Jul 14 13:59:03.698329 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:03.698302 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-dscnr" Jul 14 13:59:05.711774 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.711738 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-76zcd"] Jul 14 13:59:05.712153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.712022 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f9cf40-5020-406c-9976-e63e394014b3" containerName="extract" Jul 14 13:59:05.712153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.712032 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f9cf40-5020-406c-9976-e63e394014b3" containerName="extract" Jul 14 13:59:05.712153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.712040 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f9cf40-5020-406c-9976-e63e394014b3" containerName="pull" Jul 14 13:59:05.712153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.712045 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f9cf40-5020-406c-9976-e63e394014b3" containerName="pull" Jul 14 13:59:05.712153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.712058 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f9cf40-5020-406c-9976-e63e394014b3" containerName="util" Jul 14 13:59:05.712153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.712064 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f9cf40-5020-406c-9976-e63e394014b3" containerName="util" Jul 14 13:59:05.712153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.712122 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="36f9cf40-5020-406c-9976-e63e394014b3" containerName="extract" Jul 14 13:59:05.714384 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.714367 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-76zcd" Jul 14 13:59:05.716666 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.716650 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sptql\"" Jul 14 13:59:05.723273 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.723248 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-76zcd"] Jul 14 13:59:05.787767 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.787731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61a37a80-a9df-4307-92f9-ea1dff660278-bound-sa-token\") pod \"cert-manager-55b4d954d5-76zcd\" (UID: \"61a37a80-a9df-4307-92f9-ea1dff660278\") " pod="cert-manager/cert-manager-55b4d954d5-76zcd" Jul 14 13:59:05.787938 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.787777 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqtdn\" (UniqueName: \"kubernetes.io/projected/61a37a80-a9df-4307-92f9-ea1dff660278-kube-api-access-zqtdn\") pod \"cert-manager-55b4d954d5-76zcd\" (UID: \"61a37a80-a9df-4307-92f9-ea1dff660278\") " pod="cert-manager/cert-manager-55b4d954d5-76zcd" Jul 14 13:59:05.889065 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.889033 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61a37a80-a9df-4307-92f9-ea1dff660278-bound-sa-token\") pod \"cert-manager-55b4d954d5-76zcd\" (UID: \"61a37a80-a9df-4307-92f9-ea1dff660278\") " pod="cert-manager/cert-manager-55b4d954d5-76zcd" Jul 14 13:59:05.889216 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.889076 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqtdn\" (UniqueName: \"kubernetes.io/projected/61a37a80-a9df-4307-92f9-ea1dff660278-kube-api-access-zqtdn\") pod \"cert-manager-55b4d954d5-76zcd\" (UID: \"61a37a80-a9df-4307-92f9-ea1dff660278\") " pod="cert-manager/cert-manager-55b4d954d5-76zcd" Jul 14 13:59:05.897711 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.897680 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/61a37a80-a9df-4307-92f9-ea1dff660278-bound-sa-token\") pod \"cert-manager-55b4d954d5-76zcd\" (UID: \"61a37a80-a9df-4307-92f9-ea1dff660278\") " pod="cert-manager/cert-manager-55b4d954d5-76zcd" Jul 14 13:59:05.897824 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:05.897738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqtdn\" (UniqueName: \"kubernetes.io/projected/61a37a80-a9df-4307-92f9-ea1dff660278-kube-api-access-zqtdn\") pod \"cert-manager-55b4d954d5-76zcd\" (UID: \"61a37a80-a9df-4307-92f9-ea1dff660278\") " pod="cert-manager/cert-manager-55b4d954d5-76zcd" Jul 14 13:59:06.023887 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:06.023795 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-76zcd" Jul 14 13:59:06.162355 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:06.162323 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-76zcd"] Jul 14 13:59:06.165080 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:59:06.165055 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61a37a80_a9df_4307_92f9_ea1dff660278.slice/crio-027c8a7e70fe8c0052f78cbd477223caa176362e579039b88a74fb646638ed67 WatchSource:0}: Error finding container 027c8a7e70fe8c0052f78cbd477223caa176362e579039b88a74fb646638ed67: Status 404 returned error can't find the container with id 027c8a7e70fe8c0052f78cbd477223caa176362e579039b88a74fb646638ed67 Jul 14 13:59:06.733343 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:06.733307 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-76zcd" event={"ID":"61a37a80-a9df-4307-92f9-ea1dff660278","Type":"ContainerStarted","Data":"8bca91282d5e81826283d674f2321c75e1bbad6d7b9a8d3c582e9614e69dff1a"} Jul 14 13:59:06.733721 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:06.733356 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-76zcd" event={"ID":"61a37a80-a9df-4307-92f9-ea1dff660278","Type":"ContainerStarted","Data":"027c8a7e70fe8c0052f78cbd477223caa176362e579039b88a74fb646638ed67"} Jul 14 13:59:06.749807 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:06.749758 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-76zcd" podStartSLOduration=1.749742101 podStartE2EDuration="1.749742101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:06.748872213 +0000 UTC m=+285.975555160" watchObservedRunningTime="2026-07-14 13:59:06.749742101 +0000 UTC m=+285.976425018" Jul 14 13:59:18.468724 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.468689 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz"] Jul 14 13:59:18.471803 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.471787 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.474294 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.474259 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 13:59:18.474405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.474261 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 13:59:18.474405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.474300 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-4scxk\"" Jul 14 13:59:18.480112 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.480089 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz"] Jul 14 13:59:18.573771 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.573739 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxtjc\" (UniqueName: \"kubernetes.io/projected/437e5b51-50f4-46e4-9d5b-3b29a414b725-kube-api-access-lxtjc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.573946 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.573792 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.573946 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.573865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.674270 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.674233 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.674504 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.674393 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.674504 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.674442 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxtjc\" (UniqueName: \"kubernetes.io/projected/437e5b51-50f4-46e4-9d5b-3b29a414b725-kube-api-access-lxtjc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.674658 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.674638 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.674750 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.674732 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.685076 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.685050 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxtjc\" (UniqueName: \"kubernetes.io/projected/437e5b51-50f4-46e4-9d5b-3b29a414b725-kube-api-access-lxtjc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.781461 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.781363 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:18.913119 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:18.913067 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz"] Jul 14 13:59:18.915317 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:59:18.915292 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437e5b51_50f4_46e4_9d5b_3b29a414b725.slice/crio-3ff52032386a46dff5242d04d0493f9a57f2a83360ce02aec3481f10d110513d WatchSource:0}: Error finding container 3ff52032386a46dff5242d04d0493f9a57f2a83360ce02aec3481f10d110513d: Status 404 returned error can't find the container with id 3ff52032386a46dff5242d04d0493f9a57f2a83360ce02aec3481f10d110513d Jul 14 13:59:19.788297 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:19.788254 2564 generic.go:356] "Generic (PLEG): container finished" podID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerID="4b75f18de6627705ce292d2e12f6d0ce2948b1f142b89be04faa7aaa9fc5cd68" exitCode=0 Jul 14 13:59:19.788695 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:19.788344 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" event={"ID":"437e5b51-50f4-46e4-9d5b-3b29a414b725","Type":"ContainerDied","Data":"4b75f18de6627705ce292d2e12f6d0ce2948b1f142b89be04faa7aaa9fc5cd68"} Jul 14 13:59:19.788695 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:19.788400 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" event={"ID":"437e5b51-50f4-46e4-9d5b-3b29a414b725","Type":"ContainerStarted","Data":"3ff52032386a46dff5242d04d0493f9a57f2a83360ce02aec3481f10d110513d"} Jul 14 13:59:20.799856 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:20.799819 2564 generic.go:356] "Generic (PLEG): container finished" podID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerID="d9c054e29fc43bf1fce02994e819125a28d18d70101ed8cb00a6645cc8af519a" exitCode=0 Jul 14 13:59:20.800360 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:20.799867 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" event={"ID":"437e5b51-50f4-46e4-9d5b-3b29a414b725","Type":"ContainerDied","Data":"d9c054e29fc43bf1fce02994e819125a28d18d70101ed8cb00a6645cc8af519a"} Jul 14 13:59:21.211251 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:21.211172 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 13:59:21.214354 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:21.214330 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 13:59:21.216221 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:21.216200 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 13:59:21.805833 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:21.805796 2564 generic.go:356] "Generic (PLEG): container finished" podID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerID="62d2be0c4867aac25a507ab5ef9f59abc79a8c512c492b9abb777c58879f38bd" exitCode=0 Jul 14 13:59:21.807349 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:21.805880 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" event={"ID":"437e5b51-50f4-46e4-9d5b-3b29a414b725","Type":"ContainerDied","Data":"62d2be0c4867aac25a507ab5ef9f59abc79a8c512c492b9abb777c58879f38bd"} Jul 14 13:59:22.944601 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:22.944573 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:23.009962 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.009922 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-bundle\") pod \"437e5b51-50f4-46e4-9d5b-3b29a414b725\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " Jul 14 13:59:23.009962 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.009971 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxtjc\" (UniqueName: \"kubernetes.io/projected/437e5b51-50f4-46e4-9d5b-3b29a414b725-kube-api-access-lxtjc\") pod \"437e5b51-50f4-46e4-9d5b-3b29a414b725\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " Jul 14 13:59:23.010177 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.010018 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-util\") pod \"437e5b51-50f4-46e4-9d5b-3b29a414b725\" (UID: \"437e5b51-50f4-46e4-9d5b-3b29a414b725\") " Jul 14 13:59:23.010677 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.010647 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-bundle" (OuterVolumeSpecName: "bundle") pod "437e5b51-50f4-46e4-9d5b-3b29a414b725" (UID: "437e5b51-50f4-46e4-9d5b-3b29a414b725"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:59:23.012094 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.012066 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/437e5b51-50f4-46e4-9d5b-3b29a414b725-kube-api-access-lxtjc" (OuterVolumeSpecName: "kube-api-access-lxtjc") pod "437e5b51-50f4-46e4-9d5b-3b29a414b725" (UID: "437e5b51-50f4-46e4-9d5b-3b29a414b725"). InnerVolumeSpecName "kube-api-access-lxtjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 13:59:23.015763 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.015720 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-util" (OuterVolumeSpecName: "util") pod "437e5b51-50f4-46e4-9d5b-3b29a414b725" (UID: "437e5b51-50f4-46e4-9d5b-3b29a414b725"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 13:59:23.110905 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.110815 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-util\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:59:23.110905 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.110842 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/437e5b51-50f4-46e4-9d5b-3b29a414b725-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:59:23.110905 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.110852 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxtjc\" (UniqueName: \"kubernetes.io/projected/437e5b51-50f4-46e4-9d5b-3b29a414b725-kube-api-access-lxtjc\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 13:59:23.817138 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.817107 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" event={"ID":"437e5b51-50f4-46e4-9d5b-3b29a414b725","Type":"ContainerDied","Data":"3ff52032386a46dff5242d04d0493f9a57f2a83360ce02aec3481f10d110513d"} Jul 14 13:59:23.817326 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.817157 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afdzvgz" Jul 14 13:59:23.817326 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:23.817159 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ff52032386a46dff5242d04d0493f9a57f2a83360ce02aec3481f10d110513d" Jul 14 13:59:25.489405 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.489367 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj"] Jul 14 13:59:25.489790 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.489649 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerName="extract" Jul 14 13:59:25.489790 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.489658 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerName="extract" Jul 14 13:59:25.489790 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.489672 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerName="util" Jul 14 13:59:25.489790 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.489677 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerName="util" Jul 14 13:59:25.489790 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.489687 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerName="pull" Jul 14 13:59:25.489790 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.489695 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerName="pull" Jul 14 13:59:25.489790 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.489747 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="437e5b51-50f4-46e4-9d5b-3b29a414b725" containerName="extract" Jul 14 13:59:25.491805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.491790 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.495816 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.495794 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b4ztj\"" Jul 14 13:59:25.495979 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.495828 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 13:59:25.495979 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.495845 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 13:59:25.495979 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.495883 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 13:59:25.495979 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.495896 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 13:59:25.495979 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.495925 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 13:59:25.506246 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.506223 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj"] Jul 14 13:59:25.528966 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.528937 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d29r\" (UniqueName: \"kubernetes.io/projected/5b3fc4be-7361-4cf6-abe5-b8102de94092-kube-api-access-8d29r\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.529149 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.528976 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b3fc4be-7361-4cf6-abe5-b8102de94092-manager-config\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.529149 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.529013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b3fc4be-7361-4cf6-abe5-b8102de94092-cert\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.529149 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.529094 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b3fc4be-7361-4cf6-abe5-b8102de94092-metrics-cert\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.629928 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.629885 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d29r\" (UniqueName: \"kubernetes.io/projected/5b3fc4be-7361-4cf6-abe5-b8102de94092-kube-api-access-8d29r\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.630143 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.629941 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b3fc4be-7361-4cf6-abe5-b8102de94092-manager-config\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.630143 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.629979 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b3fc4be-7361-4cf6-abe5-b8102de94092-cert\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.630143 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.630016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b3fc4be-7361-4cf6-abe5-b8102de94092-metrics-cert\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.630753 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.630729 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5b3fc4be-7361-4cf6-abe5-b8102de94092-manager-config\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.632558 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.632530 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b3fc4be-7361-4cf6-abe5-b8102de94092-cert\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.632642 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.632603 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b3fc4be-7361-4cf6-abe5-b8102de94092-metrics-cert\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.639068 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.639036 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d29r\" (UniqueName: \"kubernetes.io/projected/5b3fc4be-7361-4cf6-abe5-b8102de94092-kube-api-access-8d29r\") pod \"lws-controller-manager-d76b86dd9-nnmjj\" (UID: \"5b3fc4be-7361-4cf6-abe5-b8102de94092\") " pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.801137 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.801110 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:25.932900 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.932875 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj"] Jul 14 13:59:25.935000 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:59:25.934967 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3fc4be_7361_4cf6_abe5_b8102de94092.slice/crio-698aaa0ccce4ea0c996e9cccff8c2de1a7be52c485367ce8e1af2b1f4924924e WatchSource:0}: Error finding container 698aaa0ccce4ea0c996e9cccff8c2de1a7be52c485367ce8e1af2b1f4924924e: Status 404 returned error can't find the container with id 698aaa0ccce4ea0c996e9cccff8c2de1a7be52c485367ce8e1af2b1f4924924e Jul 14 13:59:25.937022 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:25.937006 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 13:59:26.831884 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:26.831846 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" event={"ID":"5b3fc4be-7361-4cf6-abe5-b8102de94092","Type":"ContainerStarted","Data":"698aaa0ccce4ea0c996e9cccff8c2de1a7be52c485367ce8e1af2b1f4924924e"} Jul 14 13:59:28.843181 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:28.843144 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" event={"ID":"5b3fc4be-7361-4cf6-abe5-b8102de94092","Type":"ContainerStarted","Data":"532ae318bbe7243802abd2ebbc1b62a2937c3873cc45e917bee100013965951d"} Jul 14 13:59:28.843710 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:28.843275 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:28.926991 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:28.926932 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" podStartSLOduration=1.687596356 podStartE2EDuration="3.926915112s" totalImagesPullingTime="2.239318756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:28.926483701 +0000 UTC m=+308.153166617" watchObservedRunningTime="2026-07-14 13:59:28.926915112 +0000 UTC m=+308.153598093" Jul 14 13:59:30.611269 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.611232 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5"] Jul 14 13:59:30.614027 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.614012 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.617718 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.617694 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 13:59:30.618057 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.618039 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 13:59:30.618146 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.618106 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 13:59:30.618146 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.618126 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 13:59:30.618660 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.618647 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-654n7\"" Jul 14 13:59:30.636719 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.636686 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5"] Jul 14 13:59:30.671028 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.670999 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b14a204-1245-44b2-87b7-bf6ceb7766d5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.671198 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.671036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b14a204-1245-44b2-87b7-bf6ceb7766d5-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.671198 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.671062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nvrl\" (UniqueName: \"kubernetes.io/projected/1b14a204-1245-44b2-87b7-bf6ceb7766d5-kube-api-access-5nvrl\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.671198 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.671145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b14a204-1245-44b2-87b7-bf6ceb7766d5-tmp\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.671198 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.671190 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b14a204-1245-44b2-87b7-bf6ceb7766d5-manifests\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.772629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.772587 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b14a204-1245-44b2-87b7-bf6ceb7766d5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.772629 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.772628 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b14a204-1245-44b2-87b7-bf6ceb7766d5-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.772847 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.772657 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nvrl\" (UniqueName: \"kubernetes.io/projected/1b14a204-1245-44b2-87b7-bf6ceb7766d5-kube-api-access-5nvrl\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.772847 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.772681 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b14a204-1245-44b2-87b7-bf6ceb7766d5-tmp\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.772847 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.772709 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b14a204-1245-44b2-87b7-bf6ceb7766d5-manifests\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.773146 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.773125 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b14a204-1245-44b2-87b7-bf6ceb7766d5-manifests\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.774873 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.774846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b14a204-1245-44b2-87b7-bf6ceb7766d5-tmp\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.775073 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.775057 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b14a204-1245-44b2-87b7-bf6ceb7766d5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.775114 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.775085 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b14a204-1245-44b2-87b7-bf6ceb7766d5-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.786334 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.786313 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nvrl\" (UniqueName: \"kubernetes.io/projected/1b14a204-1245-44b2-87b7-bf6ceb7766d5-kube-api-access-5nvrl\") pod \"opendatahub-operator-controller-manager-6f858c8968-f9vs5\" (UID: \"1b14a204-1245-44b2-87b7-bf6ceb7766d5\") " pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:30.924134 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:30.924031 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:31.064987 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:31.064754 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5"] Jul 14 13:59:31.859644 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:31.859572 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" event={"ID":"1b14a204-1245-44b2-87b7-bf6ceb7766d5","Type":"ContainerStarted","Data":"16f2cfa43c1ca34cd3c012f4916c9df770903d54a20729da5646bf8a4371ad6f"} Jul 14 13:59:34.877878 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:34.877843 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" event={"ID":"1b14a204-1245-44b2-87b7-bf6ceb7766d5","Type":"ContainerStarted","Data":"03ecc61b6e4fbeef588f857baf6d69d2d92922d7546d1f4d9d50bcc525aed439"} Jul 14 13:59:35.883313 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:35.883260 2564 generic.go:356] "Generic (PLEG): container finished" podID="1b14a204-1245-44b2-87b7-bf6ceb7766d5" containerID="03ecc61b6e4fbeef588f857baf6d69d2d92922d7546d1f4d9d50bcc525aed439" exitCode=0 Jul 14 13:59:35.883805 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:35.883320 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" event={"ID":"1b14a204-1245-44b2-87b7-bf6ceb7766d5","Type":"ContainerDied","Data":"03ecc61b6e4fbeef588f857baf6d69d2d92922d7546d1f4d9d50bcc525aed439"} Jul 14 13:59:36.890027 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:36.889997 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" event={"ID":"1b14a204-1245-44b2-87b7-bf6ceb7766d5","Type":"ContainerStarted","Data":"dca7ecdc70ee1f45b959109a9916236fd74cd376632fb17b4c2b8a8b4ca30e28"} Jul 14 13:59:36.890442 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:36.890131 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:36.915740 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:36.915692 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" podStartSLOduration=2.3739470799999998 podStartE2EDuration="6.915673233s" totalImagesPullingTime="3.507839834s" totalInitContainerRuntime="1.033886319s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:36.915634733 +0000 UTC m=+316.142317708" watchObservedRunningTime="2026-07-14 13:59:36.915673233 +0000 UTC m=+316.142356148" Jul 14 13:59:39.849807 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:39.849777 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d76b86dd9-nnmjj" Jul 14 13:59:47.898437 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:47.898408 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f858c8968-f9vs5" Jul 14 13:59:51.521275 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.521217 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nnfjj"] Jul 14 13:59:51.528191 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.528171 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:51.530833 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.530808 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 13:59:51.530956 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.530849 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-kjctl\"" Jul 14 13:59:51.535560 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.535427 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nnfjj"] Jul 14 13:59:51.653666 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.653621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gwgm\" (UniqueName: \"kubernetes.io/projected/8464a10c-0d95-42e7-9650-c644e99ed335-kube-api-access-4gwgm\") pod \"model-serving-api-54d89bd5fc-nnfjj\" (UID: \"8464a10c-0d95-42e7-9650-c644e99ed335\") " pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:51.653860 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.653771 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8464a10c-0d95-42e7-9650-c644e99ed335-tls-certs\") pod \"model-serving-api-54d89bd5fc-nnfjj\" (UID: \"8464a10c-0d95-42e7-9650-c644e99ed335\") " pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:51.754462 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.754425 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8464a10c-0d95-42e7-9650-c644e99ed335-tls-certs\") pod \"model-serving-api-54d89bd5fc-nnfjj\" (UID: \"8464a10c-0d95-42e7-9650-c644e99ed335\") " pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:51.754631 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.754469 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gwgm\" (UniqueName: \"kubernetes.io/projected/8464a10c-0d95-42e7-9650-c644e99ed335-kube-api-access-4gwgm\") pod \"model-serving-api-54d89bd5fc-nnfjj\" (UID: \"8464a10c-0d95-42e7-9650-c644e99ed335\") " pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:51.756845 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.756809 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8464a10c-0d95-42e7-9650-c644e99ed335-tls-certs\") pod \"model-serving-api-54d89bd5fc-nnfjj\" (UID: \"8464a10c-0d95-42e7-9650-c644e99ed335\") " pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:51.763743 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.763723 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gwgm\" (UniqueName: \"kubernetes.io/projected/8464a10c-0d95-42e7-9650-c644e99ed335-kube-api-access-4gwgm\") pod \"model-serving-api-54d89bd5fc-nnfjj\" (UID: \"8464a10c-0d95-42e7-9650-c644e99ed335\") " pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:51.839277 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.839251 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:51.972865 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:51.972842 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nnfjj"] Jul 14 13:59:51.974651 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:59:51.974626 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8464a10c_0d95_42e7_9650_c644e99ed335.slice/crio-b88d03ca0a0cf6f2e12c57b063406befc5b23077ffdabf294cd26d466c046c57 WatchSource:0}: Error finding container b88d03ca0a0cf6f2e12c57b063406befc5b23077ffdabf294cd26d466c046c57: Status 404 returned error can't find the container with id b88d03ca0a0cf6f2e12c57b063406befc5b23077ffdabf294cd26d466c046c57 Jul 14 13:59:52.976033 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:52.975986 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" event={"ID":"8464a10c-0d95-42e7-9650-c644e99ed335","Type":"ContainerStarted","Data":"b88d03ca0a0cf6f2e12c57b063406befc5b23077ffdabf294cd26d466c046c57"} Jul 14 13:59:54.987796 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:54.987757 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" event={"ID":"8464a10c-0d95-42e7-9650-c644e99ed335","Type":"ContainerStarted","Data":"e19bd3b96b113fcb7c36f4a2933ffc2036e3a85137776e92644661f88caa9aa8"} Jul 14 13:59:54.988232 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:54.987864 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 13:59:55.006246 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:55.006196 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" podStartSLOduration=2.012947795 podStartE2EDuration="4.006181013s" totalImagesPullingTime="1.993233218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 13:59:55.005692288 +0000 UTC m=+334.232375235" watchObservedRunningTime="2026-07-14 13:59:55.006181013 +0000 UTC m=+334.232863930" Jul 14 13:59:58.007260 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.007232 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn"] Jul 14 13:59:58.009452 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.009436 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.012411 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.012387 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 13:59:58.012915 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.012898 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-nn7lb\"" Jul 14 13:59:58.013169 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.013149 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 13:59:58.013432 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.013414 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 13:59:58.026244 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.026211 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn"] Jul 14 13:59:58.116419 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.116419 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116419 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxczv\" (UniqueName: \"kubernetes.io/projected/d7e5c913-2cb3-4baf-a74e-e515c783803f-kube-api-access-kxczv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.116676 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116453 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e5c913-2cb3-4baf-a74e-e515c783803f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.116676 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116522 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.116676 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116592 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.116676 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.116676 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116654 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.116976 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116685 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.116976 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.116711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.175000 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.174964 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh"] Jul 14 13:59:58.178700 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.178682 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.181803 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.181780 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 13:59:58.181923 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.181778 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 13:59:58.181923 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.181785 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 13:59:58.181923 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.181830 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-czzr5\"" Jul 14 13:59:58.195741 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.195717 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh"] Jul 14 13:59:58.217232 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217403 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217403 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217403 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217403 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217403 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217394 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217641 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217443 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217641 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217467 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxczv\" (UniqueName: \"kubernetes.io/projected/d7e5c913-2cb3-4baf-a74e-e515c783803f-kube-api-access-kxczv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217641 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e5c913-2cb3-4baf-a74e-e515c783803f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217795 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217767 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217866 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217842 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.217914 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.217897 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.218103 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.218085 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.218339 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.218318 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e5c913-2cb3-4baf-a74e-e515c783803f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.219721 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.219698 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.219825 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.219789 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.238226 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.238202 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e5c913-2cb3-4baf-a74e-e515c783803f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.243110 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.243083 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxczv\" (UniqueName: \"kubernetes.io/projected/d7e5c913-2cb3-4baf-a74e-e515c783803f-kube-api-access-kxczv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4ld2rn\" (UID: \"d7e5c913-2cb3-4baf-a74e-e515c783803f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.317995 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.317962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk92t\" (UniqueName: \"kubernetes.io/projected/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-kube-api-access-jk92t\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.318153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.318010 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.318153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.318077 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.318153 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.318108 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.318258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.318157 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.318258 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.318222 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.318354 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.318270 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.319924 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.319905 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 13:59:58.419057 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.419019 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.419057 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.419059 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk92t\" (UniqueName: \"kubernetes.io/projected/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-kube-api-access-jk92t\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.419325 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.419091 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.419325 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.419116 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.419325 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.419134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.419325 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.419163 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.419325 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.419196 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.420136 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.420025 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.421780 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.421748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.421879 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.421797 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.421967 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.421953 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.422004 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.421975 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.444087 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.444060 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.455438 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.455410 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk92t\" (UniqueName: \"kubernetes.io/projected/3f90acdb-1e88-47af-9e32-b1cb47ae4b62-kube-api-access-jk92t\") pod \"istiod-openshift-gateway-94bb8fbfd-pvhmh\" (UID: \"3f90acdb-1e88-47af-9e32-b1cb47ae4b62\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.479532 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.479507 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn"] Jul 14 13:59:58.481134 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:59:58.481106 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e5c913_2cb3_4baf_a74e_e515c783803f.slice/crio-64bcd3f902c197574f808bc1bb7c73fbb499678be37f6e8c00890be4479363b2 WatchSource:0}: Error finding container 64bcd3f902c197574f808bc1bb7c73fbb499678be37f6e8c00890be4479363b2: Status 404 returned error can't find the container with id 64bcd3f902c197574f808bc1bb7c73fbb499678be37f6e8c00890be4479363b2 Jul 14 13:59:58.487602 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.487583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 13:59:58.638630 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:58.638604 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh"] Jul 14 13:59:58.640596 ip-10-0-142-141 kubenswrapper[2564]: W0714 13:59:58.640568 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f90acdb_1e88_47af_9e32_b1cb47ae4b62.slice/crio-12f79ca18adb68932d9a69bfae2e9e17ad1a1ce420509d83d332f88dc72a0f8c WatchSource:0}: Error finding container 12f79ca18adb68932d9a69bfae2e9e17ad1a1ce420509d83d332f88dc72a0f8c: Status 404 returned error can't find the container with id 12f79ca18adb68932d9a69bfae2e9e17ad1a1ce420509d83d332f88dc72a0f8c Jul 14 13:59:59.009160 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:59.009057 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" event={"ID":"d7e5c913-2cb3-4baf-a74e-e515c783803f","Type":"ContainerStarted","Data":"64bcd3f902c197574f808bc1bb7c73fbb499678be37f6e8c00890be4479363b2"} Jul 14 13:59:59.011275 ip-10-0-142-141 kubenswrapper[2564]: I0714 13:59:59.011242 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" event={"ID":"3f90acdb-1e88-47af-9e32-b1cb47ae4b62","Type":"ContainerStarted","Data":"12f79ca18adb68932d9a69bfae2e9e17ad1a1ce420509d83d332f88dc72a0f8c"} Jul 14 14:00:02.065606 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:02.065569 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:00:02.065887 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:02.065639 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:00:02.065887 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:02.065669 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:00:02.066726 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:02.066693 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:00:02.066815 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:02.066756 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:00:03.033315 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:03.033259 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" event={"ID":"3f90acdb-1e88-47af-9e32-b1cb47ae4b62","Type":"ContainerStarted","Data":"b245a8789c2d62790509f0b99a74ba254e1b7adb5174dfbe8579bb1c6922326a"} Jul 14 14:00:03.033490 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:03.033353 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 14:00:03.034976 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:03.034949 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" event={"ID":"d7e5c913-2cb3-4baf-a74e-e515c783803f","Type":"ContainerStarted","Data":"a8a483b352b43bca855cc139b3da5adc20ad2abd962282869a85b858b8cb4872"} Jul 14 14:00:03.035181 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:03.035167 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" Jul 14 14:00:03.079982 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:03.079926 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pvhmh" podStartSLOduration=1.656088908 podStartE2EDuration="5.079907914s" totalImagesPullingTime="3.423819006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:00:03.077885149 +0000 UTC m=+342.304568093" watchObservedRunningTime="2026-07-14 14:00:03.079907914 +0000 UTC m=+342.306590832" Jul 14 14:00:03.111703 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:03.111651 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" podStartSLOduration=2.52953024 podStartE2EDuration="6.111633653s" totalImagesPullingTime="3.582103413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 13:59:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:00:03.109348616 +0000 UTC m=+342.336031550" watchObservedRunningTime="2026-07-14 14:00:03.111633653 +0000 UTC m=+342.338316606" Jul 14 14:00:03.320356 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:03.320315 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 14:00:03.324944 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:03.324918 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 14:00:04.040358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:04.040322 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 14:00:04.041324 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:04.041301 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4ld2rn" Jul 14 14:00:05.995997 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:00:05.995965 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-nnfjj" Jul 14 14:01:06.855735 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.855694 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn"] Jul 14 14:01:06.860202 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.860179 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:06.863826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.863802 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 14:01:06.864218 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.864203 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ncn5b\"" Jul 14 14:01:06.865120 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.865105 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 14:01:06.869758 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.869717 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn"] Jul 14 14:01:06.897521 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.897490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:06.897678 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.897539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:06.897678 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.897625 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpwwv\" (UniqueName: \"kubernetes.io/projected/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-kube-api-access-kpwwv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:06.998792 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.998753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:06.998960 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.998806 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:06.998960 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.998853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpwwv\" (UniqueName: \"kubernetes.io/projected/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-kube-api-access-kpwwv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:06.999177 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.999160 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:06.999223 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:06.999201 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:07.007219 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.007185 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpwwv\" (UniqueName: \"kubernetes.io/projected/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-kube-api-access-kpwwv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:07.169431 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.169347 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:07.304516 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.304489 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn"] Jul 14 14:01:07.307171 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:01:07.307148 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aac65cf_be49_49c3_b9dc_c61fb45b4fb8.slice/crio-a143b5b69f1189a140d832047d1f9af435d0c79e3deff90decfd37c44693d024 WatchSource:0}: Error finding container a143b5b69f1189a140d832047d1f9af435d0c79e3deff90decfd37c44693d024: Status 404 returned error can't find the container with id a143b5b69f1189a140d832047d1f9af435d0c79e3deff90decfd37c44693d024 Jul 14 14:01:07.323985 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.323955 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" event={"ID":"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8","Type":"ContainerStarted","Data":"a143b5b69f1189a140d832047d1f9af435d0c79e3deff90decfd37c44693d024"} Jul 14 14:01:07.658810 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.658773 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw"] Jul 14 14:01:07.661302 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.661267 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.671730 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.671668 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw"] Jul 14 14:01:07.705373 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.705340 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfbc2\" (UniqueName: \"kubernetes.io/projected/3709c140-e4a7-4645-abbf-f5710a658197-kube-api-access-pfbc2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.705590 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.705393 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.705590 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.705483 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.806134 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.806093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.806358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.806188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfbc2\" (UniqueName: \"kubernetes.io/projected/3709c140-e4a7-4645-abbf-f5710a658197-kube-api-access-pfbc2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.806358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.806229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.806596 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.806560 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.806663 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.806598 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.814922 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.814895 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfbc2\" (UniqueName: \"kubernetes.io/projected/3709c140-e4a7-4645-abbf-f5710a658197-kube-api-access-pfbc2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:07.970522 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:07.970426 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:08.056747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.056716 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk"] Jul 14 14:01:08.060107 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.060084 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.069788 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.069762 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk"] Jul 14 14:01:08.104124 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.104100 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw"] Jul 14 14:01:08.105889 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:01:08.105865 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3709c140_e4a7_4645_abbf_f5710a658197.slice/crio-bda395334d5303c8313fe58497729faae291b53e721f0cb8c7d538e9d9966259 WatchSource:0}: Error finding container bda395334d5303c8313fe58497729faae291b53e721f0cb8c7d538e9d9966259: Status 404 returned error can't find the container with id bda395334d5303c8313fe58497729faae291b53e721f0cb8c7d538e9d9966259 Jul 14 14:01:08.109403 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.109376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.109529 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.109456 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.109529 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.109524 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2jj9\" (UniqueName: \"kubernetes.io/projected/ecef76ae-1f89-4b0e-8f16-27de8d969c11-kube-api-access-v2jj9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.210374 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.210344 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.210502 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.210426 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2jj9\" (UniqueName: \"kubernetes.io/projected/ecef76ae-1f89-4b0e-8f16-27de8d969c11-kube-api-access-v2jj9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.210569 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.210541 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.210747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.210725 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.210850 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.210830 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.218940 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.218918 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2jj9\" (UniqueName: \"kubernetes.io/projected/ecef76ae-1f89-4b0e-8f16-27de8d969c11-kube-api-access-v2jj9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.335745 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.335710 2564 generic.go:356] "Generic (PLEG): container finished" podID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerID="b0c7ddd9e80a1f7cd97a434e3dfe2e1dcd3067a55e4a34deefcb6c92dd134c58" exitCode=0 Jul 14 14:01:08.335947 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.335797 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" event={"ID":"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8","Type":"ContainerDied","Data":"b0c7ddd9e80a1f7cd97a434e3dfe2e1dcd3067a55e4a34deefcb6c92dd134c58"} Jul 14 14:01:08.337573 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.337553 2564 generic.go:356] "Generic (PLEG): container finished" podID="3709c140-e4a7-4645-abbf-f5710a658197" containerID="3f48ffb1e06daf6d3c9ba0f15208c2012cb77753f229b1234cded6d4ea6b117e" exitCode=0 Jul 14 14:01:08.337663 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.337604 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" event={"ID":"3709c140-e4a7-4645-abbf-f5710a658197","Type":"ContainerDied","Data":"3f48ffb1e06daf6d3c9ba0f15208c2012cb77753f229b1234cded6d4ea6b117e"} Jul 14 14:01:08.337663 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.337631 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" event={"ID":"3709c140-e4a7-4645-abbf-f5710a658197","Type":"ContainerStarted","Data":"bda395334d5303c8313fe58497729faae291b53e721f0cb8c7d538e9d9966259"} Jul 14 14:01:08.372788 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.372756 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:08.449111 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.449074 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq"] Jul 14 14:01:08.451792 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.451763 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.463581 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.463552 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq"] Jul 14 14:01:08.509697 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.509674 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk"] Jul 14 14:01:08.511638 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:01:08.511614 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecef76ae_1f89_4b0e_8f16_27de8d969c11.slice/crio-a49e70fa72b912cf8efe3c7f8d18077d37071248729038798534b6ea1d8e0534 WatchSource:0}: Error finding container a49e70fa72b912cf8efe3c7f8d18077d37071248729038798534b6ea1d8e0534: Status 404 returned error can't find the container with id a49e70fa72b912cf8efe3c7f8d18077d37071248729038798534b6ea1d8e0534 Jul 14 14:01:08.513339 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.513321 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.513431 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.513368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ttsz\" (UniqueName: \"kubernetes.io/projected/9a56ee5f-38c2-4d6e-86f6-9440960eb188-kube-api-access-9ttsz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.513431 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.513394 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.614916 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.614841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.614916 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.614900 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ttsz\" (UniqueName: \"kubernetes.io/projected/9a56ee5f-38c2-4d6e-86f6-9440960eb188-kube-api-access-9ttsz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.615106 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.614925 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.615237 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.615210 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.615325 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.615261 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.623936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.623912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ttsz\" (UniqueName: \"kubernetes.io/projected/9a56ee5f-38c2-4d6e-86f6-9440960eb188-kube-api-access-9ttsz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.765708 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.765673 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:08.895838 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:08.895810 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq"] Jul 14 14:01:08.897135 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:01:08.897104 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a56ee5f_38c2_4d6e_86f6_9440960eb188.slice/crio-57de73c37f7ac4206c90609cb50b58903fc5c567c94368dd332a62ee8b1797e0 WatchSource:0}: Error finding container 57de73c37f7ac4206c90609cb50b58903fc5c567c94368dd332a62ee8b1797e0: Status 404 returned error can't find the container with id 57de73c37f7ac4206c90609cb50b58903fc5c567c94368dd332a62ee8b1797e0 Jul 14 14:01:09.347447 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:09.347412 2564 generic.go:356] "Generic (PLEG): container finished" podID="3709c140-e4a7-4645-abbf-f5710a658197" containerID="e3719f2a2a5a426435ba7fc404efaa2bb0cec1a064b8509d1d286a6b005911bf" exitCode=0 Jul 14 14:01:09.347860 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:09.347487 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" event={"ID":"3709c140-e4a7-4645-abbf-f5710a658197","Type":"ContainerDied","Data":"e3719f2a2a5a426435ba7fc404efaa2bb0cec1a064b8509d1d286a6b005911bf"} Jul 14 14:01:09.349029 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:09.349005 2564 generic.go:356] "Generic (PLEG): container finished" podID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerID="0bf8af892a8c643c226eb03c56832b2096880a33ec81683fa65add7b78877adf" exitCode=0 Jul 14 14:01:09.349126 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:09.349048 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" event={"ID":"9a56ee5f-38c2-4d6e-86f6-9440960eb188","Type":"ContainerDied","Data":"0bf8af892a8c643c226eb03c56832b2096880a33ec81683fa65add7b78877adf"} Jul 14 14:01:09.349126 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:09.349094 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" event={"ID":"9a56ee5f-38c2-4d6e-86f6-9440960eb188","Type":"ContainerStarted","Data":"57de73c37f7ac4206c90609cb50b58903fc5c567c94368dd332a62ee8b1797e0"} Jul 14 14:01:09.350616 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:09.350596 2564 generic.go:356] "Generic (PLEG): container finished" podID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerID="7531e12a6aa2f8ffeed281c385953eba5b0600f8f8dfff2f5bc0dc04f93e1ced" exitCode=0 Jul 14 14:01:09.350722 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:09.350657 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" event={"ID":"ecef76ae-1f89-4b0e-8f16-27de8d969c11","Type":"ContainerDied","Data":"7531e12a6aa2f8ffeed281c385953eba5b0600f8f8dfff2f5bc0dc04f93e1ced"} Jul 14 14:01:09.350722 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:09.350693 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" event={"ID":"ecef76ae-1f89-4b0e-8f16-27de8d969c11","Type":"ContainerStarted","Data":"a49e70fa72b912cf8efe3c7f8d18077d37071248729038798534b6ea1d8e0534"} Jul 14 14:01:10.357572 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:10.357536 2564 generic.go:356] "Generic (PLEG): container finished" podID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerID="245e33f860cf449f57597b771b7420e08d59cd5ab4a3b17623ab05c9e894bae3" exitCode=0 Jul 14 14:01:10.358039 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:10.357578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" event={"ID":"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8","Type":"ContainerDied","Data":"245e33f860cf449f57597b771b7420e08d59cd5ab4a3b17623ab05c9e894bae3"} Jul 14 14:01:10.359784 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:10.359764 2564 generic.go:356] "Generic (PLEG): container finished" podID="3709c140-e4a7-4645-abbf-f5710a658197" containerID="4b33761f906f33434651245d2a9f754560602a3b8abfaa36f17a833380353487" exitCode=0 Jul 14 14:01:10.359873 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:10.359812 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" event={"ID":"3709c140-e4a7-4645-abbf-f5710a658197","Type":"ContainerDied","Data":"4b33761f906f33434651245d2a9f754560602a3b8abfaa36f17a833380353487"} Jul 14 14:01:11.375966 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.375932 2564 generic.go:356] "Generic (PLEG): container finished" podID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerID="f602ede8c3801210a378f01bd8fad069bc27f70bcd69e40cabfbfcc23135f2cb" exitCode=0 Jul 14 14:01:11.377773 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.377743 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" event={"ID":"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8","Type":"ContainerDied","Data":"f602ede8c3801210a378f01bd8fad069bc27f70bcd69e40cabfbfcc23135f2cb"} Jul 14 14:01:11.377980 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.377956 2564 generic.go:356] "Generic (PLEG): container finished" podID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerID="1670c8c080809de5486677c397379292a23a71409930fe1d9b6f09cf2627addd" exitCode=0 Jul 14 14:01:11.378056 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.378037 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" event={"ID":"9a56ee5f-38c2-4d6e-86f6-9440960eb188","Type":"ContainerDied","Data":"1670c8c080809de5486677c397379292a23a71409930fe1d9b6f09cf2627addd"} Jul 14 14:01:11.379864 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.379844 2564 generic.go:356] "Generic (PLEG): container finished" podID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerID="33aa55430fd80277656a1151af5c263ed55f382ed9bb2579d475f4379aa6f777" exitCode=0 Jul 14 14:01:11.379949 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.379873 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" event={"ID":"ecef76ae-1f89-4b0e-8f16-27de8d969c11","Type":"ContainerDied","Data":"33aa55430fd80277656a1151af5c263ed55f382ed9bb2579d475f4379aa6f777"} Jul 14 14:01:11.518947 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.518919 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:11.540275 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.540248 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-util\") pod \"3709c140-e4a7-4645-abbf-f5710a658197\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " Jul 14 14:01:11.540436 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.540336 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfbc2\" (UniqueName: \"kubernetes.io/projected/3709c140-e4a7-4645-abbf-f5710a658197-kube-api-access-pfbc2\") pod \"3709c140-e4a7-4645-abbf-f5710a658197\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " Jul 14 14:01:11.540436 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.540387 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-bundle\") pod \"3709c140-e4a7-4645-abbf-f5710a658197\" (UID: \"3709c140-e4a7-4645-abbf-f5710a658197\") " Jul 14 14:01:11.541173 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.541144 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-bundle" (OuterVolumeSpecName: "bundle") pod "3709c140-e4a7-4645-abbf-f5710a658197" (UID: "3709c140-e4a7-4645-abbf-f5710a658197"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:01:11.542920 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.542895 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3709c140-e4a7-4645-abbf-f5710a658197-kube-api-access-pfbc2" (OuterVolumeSpecName: "kube-api-access-pfbc2") pod "3709c140-e4a7-4645-abbf-f5710a658197" (UID: "3709c140-e4a7-4645-abbf-f5710a658197"). InnerVolumeSpecName "kube-api-access-pfbc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:01:11.547261 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.547236 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-util" (OuterVolumeSpecName: "util") pod "3709c140-e4a7-4645-abbf-f5710a658197" (UID: "3709c140-e4a7-4645-abbf-f5710a658197"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:01:11.641689 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.641655 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-util\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:11.641689 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.641686 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfbc2\" (UniqueName: \"kubernetes.io/projected/3709c140-e4a7-4645-abbf-f5710a658197-kube-api-access-pfbc2\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:11.641864 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:11.641697 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3709c140-e4a7-4645-abbf-f5710a658197-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:12.388256 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.388173 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" event={"ID":"3709c140-e4a7-4645-abbf-f5710a658197","Type":"ContainerDied","Data":"bda395334d5303c8313fe58497729faae291b53e721f0cb8c7d538e9d9966259"} Jul 14 14:01:12.388256 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.388218 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bda395334d5303c8313fe58497729faae291b53e721f0cb8c7d538e9d9966259" Jul 14 14:01:12.388256 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.388187 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw" Jul 14 14:01:12.390394 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.390369 2564 generic.go:356] "Generic (PLEG): container finished" podID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerID="4e487bae14b90989d8e4fda30dbb55f0630d2cd0669b60fa17bbf6bbfb3f425a" exitCode=0 Jul 14 14:01:12.390496 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.390435 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" event={"ID":"9a56ee5f-38c2-4d6e-86f6-9440960eb188","Type":"ContainerDied","Data":"4e487bae14b90989d8e4fda30dbb55f0630d2cd0669b60fa17bbf6bbfb3f425a"} Jul 14 14:01:12.392461 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.392438 2564 generic.go:356] "Generic (PLEG): container finished" podID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerID="52d84b75d668704339bdf90f11db0fe1cd820493bf7a7555c5e888e87034d481" exitCode=0 Jul 14 14:01:12.392552 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.392522 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" event={"ID":"ecef76ae-1f89-4b0e-8f16-27de8d969c11","Type":"ContainerDied","Data":"52d84b75d668704339bdf90f11db0fe1cd820493bf7a7555c5e888e87034d481"} Jul 14 14:01:12.527421 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.527398 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:12.550227 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.550195 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-util\") pod \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " Jul 14 14:01:12.550436 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.550322 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpwwv\" (UniqueName: \"kubernetes.io/projected/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-kube-api-access-kpwwv\") pod \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " Jul 14 14:01:12.550436 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.550362 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-bundle\") pod \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\" (UID: \"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8\") " Jul 14 14:01:12.551031 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.551003 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-bundle" (OuterVolumeSpecName: "bundle") pod "2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" (UID: "2aac65cf-be49-49c3-b9dc-c61fb45b4fb8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:01:12.552638 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.552609 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-kube-api-access-kpwwv" (OuterVolumeSpecName: "kube-api-access-kpwwv") pod "2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" (UID: "2aac65cf-be49-49c3-b9dc-c61fb45b4fb8"). InnerVolumeSpecName "kube-api-access-kpwwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:01:12.579879 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.579841 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-util" (OuterVolumeSpecName: "util") pod "2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" (UID: "2aac65cf-be49-49c3-b9dc-c61fb45b4fb8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:01:12.651798 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.651708 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpwwv\" (UniqueName: \"kubernetes.io/projected/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-kube-api-access-kpwwv\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:12.651798 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.651738 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:12.651798 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:12.651752 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aac65cf-be49-49c3-b9dc-c61fb45b4fb8-util\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:13.398882 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.398850 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" Jul 14 14:01:13.398882 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.398879 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn" event={"ID":"2aac65cf-be49-49c3-b9dc-c61fb45b4fb8","Type":"ContainerDied","Data":"a143b5b69f1189a140d832047d1f9af435d0c79e3deff90decfd37c44693d024"} Jul 14 14:01:13.399309 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.398921 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a143b5b69f1189a140d832047d1f9af435d0c79e3deff90decfd37c44693d024" Jul 14 14:01:13.551972 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.551945 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:13.555183 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.555162 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:13.660041 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.659936 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ttsz\" (UniqueName: \"kubernetes.io/projected/9a56ee5f-38c2-4d6e-86f6-9440960eb188-kube-api-access-9ttsz\") pod \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " Jul 14 14:01:13.660041 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.659995 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-util\") pod \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " Jul 14 14:01:13.660041 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.660020 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-util\") pod \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " Jul 14 14:01:13.660355 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.660127 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-bundle\") pod \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " Jul 14 14:01:13.660355 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.660158 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-bundle\") pod \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\" (UID: \"9a56ee5f-38c2-4d6e-86f6-9440960eb188\") " Jul 14 14:01:13.660355 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.660221 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2jj9\" (UniqueName: \"kubernetes.io/projected/ecef76ae-1f89-4b0e-8f16-27de8d969c11-kube-api-access-v2jj9\") pod \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\" (UID: \"ecef76ae-1f89-4b0e-8f16-27de8d969c11\") " Jul 14 14:01:13.660724 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.660692 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-bundle" (OuterVolumeSpecName: "bundle") pod "9a56ee5f-38c2-4d6e-86f6-9440960eb188" (UID: "9a56ee5f-38c2-4d6e-86f6-9440960eb188"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:01:13.660843 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.660788 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-bundle" (OuterVolumeSpecName: "bundle") pod "ecef76ae-1f89-4b0e-8f16-27de8d969c11" (UID: "ecef76ae-1f89-4b0e-8f16-27de8d969c11"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:01:13.662236 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.662205 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a56ee5f-38c2-4d6e-86f6-9440960eb188-kube-api-access-9ttsz" (OuterVolumeSpecName: "kube-api-access-9ttsz") pod "9a56ee5f-38c2-4d6e-86f6-9440960eb188" (UID: "9a56ee5f-38c2-4d6e-86f6-9440960eb188"). InnerVolumeSpecName "kube-api-access-9ttsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:01:13.662654 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.662627 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecef76ae-1f89-4b0e-8f16-27de8d969c11-kube-api-access-v2jj9" (OuterVolumeSpecName: "kube-api-access-v2jj9") pod "ecef76ae-1f89-4b0e-8f16-27de8d969c11" (UID: "ecef76ae-1f89-4b0e-8f16-27de8d969c11"). InnerVolumeSpecName "kube-api-access-v2jj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:01:13.665826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.665795 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-util" (OuterVolumeSpecName: "util") pod "9a56ee5f-38c2-4d6e-86f6-9440960eb188" (UID: "9a56ee5f-38c2-4d6e-86f6-9440960eb188"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:01:13.666633 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.666614 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-util" (OuterVolumeSpecName: "util") pod "ecef76ae-1f89-4b0e-8f16-27de8d969c11" (UID: "ecef76ae-1f89-4b0e-8f16-27de8d969c11"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:01:13.761117 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.761075 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ttsz\" (UniqueName: \"kubernetes.io/projected/9a56ee5f-38c2-4d6e-86f6-9440960eb188-kube-api-access-9ttsz\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:13.761117 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.761108 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-util\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:13.761117 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.761120 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-util\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:13.761117 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.761128 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ecef76ae-1f89-4b0e-8f16-27de8d969c11-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:13.761424 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.761138 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a56ee5f-38c2-4d6e-86f6-9440960eb188-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:13.761424 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:13.761146 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2jj9\" (UniqueName: \"kubernetes.io/projected/ecef76ae-1f89-4b0e-8f16-27de8d969c11-kube-api-access-v2jj9\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:01:14.405540 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:14.405507 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" Jul 14 14:01:14.405966 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:14.405504 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq" event={"ID":"9a56ee5f-38c2-4d6e-86f6-9440960eb188","Type":"ContainerDied","Data":"57de73c37f7ac4206c90609cb50b58903fc5c567c94368dd332a62ee8b1797e0"} Jul 14 14:01:14.405966 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:14.405623 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57de73c37f7ac4206c90609cb50b58903fc5c567c94368dd332a62ee8b1797e0" Jul 14 14:01:14.407423 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:14.407401 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" Jul 14 14:01:14.407423 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:14.407411 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk" event={"ID":"ecef76ae-1f89-4b0e-8f16-27de8d969c11","Type":"ContainerDied","Data":"a49e70fa72b912cf8efe3c7f8d18077d37071248729038798534b6ea1d8e0534"} Jul 14 14:01:14.407568 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:01:14.407447 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a49e70fa72b912cf8efe3c7f8d18077d37071248729038798534b6ea1d8e0534" Jul 14 14:02:02.946830 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:02.946749 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f76975456-27tr6"] Jul 14 14:02:22.617803 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.617767 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f"] Jul 14 14:02:22.618305 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618263 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerName="pull" Jul 14 14:02:22.618305 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618302 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerName="pull" Jul 14 14:02:22.618388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618316 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerName="pull" Jul 14 14:02:22.618388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618325 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerName="pull" Jul 14 14:02:22.618388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618345 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerName="util" Jul 14 14:02:22.618388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618354 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerName="util" Jul 14 14:02:22.618388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618364 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerName="extract" Jul 14 14:02:22.618388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618372 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerName="extract" Jul 14 14:02:22.618388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618389 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3709c140-e4a7-4645-abbf-f5710a658197" containerName="extract" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618397 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3709c140-e4a7-4645-abbf-f5710a658197" containerName="extract" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618415 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerName="extract" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618423 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerName="extract" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618433 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerName="extract" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618441 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerName="extract" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618453 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3709c140-e4a7-4645-abbf-f5710a658197" containerName="util" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618461 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3709c140-e4a7-4645-abbf-f5710a658197" containerName="util" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618472 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3709c140-e4a7-4645-abbf-f5710a658197" containerName="pull" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618479 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3709c140-e4a7-4645-abbf-f5710a658197" containerName="pull" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618491 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerName="pull" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618499 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerName="pull" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618516 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerName="util" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618524 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerName="util" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618536 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerName="util" Jul 14 14:02:22.618587 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618544 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerName="util" Jul 14 14:02:22.619003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618627 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3709c140-e4a7-4645-abbf-f5710a658197" containerName="extract" Jul 14 14:02:22.619003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618640 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2aac65cf-be49-49c3-b9dc-c61fb45b4fb8" containerName="extract" Jul 14 14:02:22.619003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618650 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecef76ae-1f89-4b0e-8f16-27de8d969c11" containerName="extract" Jul 14 14:02:22.619003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.618661 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a56ee5f-38c2-4d6e-86f6-9440960eb188" containerName="extract" Jul 14 14:02:22.620820 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.620803 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.623846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.623825 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ckzjk\"" Jul 14 14:02:22.634086 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.634063 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f"] Jul 14 14:02:22.746764 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.746732 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.746764 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.746768 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.747020 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.746788 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/027a7ee3-a371-4302-936c-dd3a282705cf-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.747020 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.746809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.747020 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.746901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/027a7ee3-a371-4302-936c-dd3a282705cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.747020 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.746982 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/027a7ee3-a371-4302-936c-dd3a282705cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.747211 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.747024 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.747211 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.747050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56g2x\" (UniqueName: \"kubernetes.io/projected/027a7ee3-a371-4302-936c-dd3a282705cf-kube-api-access-56g2x\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.747211 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.747078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848134 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848345 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848168 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848345 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848215 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/027a7ee3-a371-4302-936c-dd3a282705cf-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848345 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848245 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848345 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848320 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/027a7ee3-a371-4302-936c-dd3a282705cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848577 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/027a7ee3-a371-4302-936c-dd3a282705cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848577 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848409 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848577 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56g2x\" (UniqueName: \"kubernetes.io/projected/027a7ee3-a371-4302-936c-dd3a282705cf-kube-api-access-56g2x\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848577 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848458 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848765 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848704 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.848895 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.848872 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.849194 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.849170 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.849194 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.849184 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.849689 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.849667 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/027a7ee3-a371-4302-936c-dd3a282705cf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.850678 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.850656 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/027a7ee3-a371-4302-936c-dd3a282705cf-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.851466 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.851448 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/027a7ee3-a371-4302-936c-dd3a282705cf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.875299 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.875212 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/027a7ee3-a371-4302-936c-dd3a282705cf-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.877121 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.877100 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56g2x\" (UniqueName: \"kubernetes.io/projected/027a7ee3-a371-4302-936c-dd3a282705cf-kube-api-access-56g2x\") pod \"maas-default-gateway-openshift-default-56654dcf7-4758f\" (UID: \"027a7ee3-a371-4302-936c-dd3a282705cf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:22.930477 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:22.930453 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:23.079462 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:23.079438 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f"] Jul 14 14:02:23.081272 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:02:23.081251 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027a7ee3_a371_4302_936c_dd3a282705cf.slice/crio-a10b936954708e58a709cb609078ce2e446b003bd4e965d7fbf366510c81ab45 WatchSource:0}: Error finding container a10b936954708e58a709cb609078ce2e446b003bd4e965d7fbf366510c81ab45: Status 404 returned error can't find the container with id a10b936954708e58a709cb609078ce2e446b003bd4e965d7fbf366510c81ab45 Jul 14 14:02:23.083267 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:23.083236 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:02:23.083388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:23.083324 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:02:23.083388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:23.083373 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:02:23.750937 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:23.750901 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" event={"ID":"027a7ee3-a371-4302-936c-dd3a282705cf","Type":"ContainerStarted","Data":"b3507b0327560ec22fa2cd1830c2d5bde2f4463a91effd1992af983b873e6c01"} Jul 14 14:02:23.751375 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:23.750956 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" event={"ID":"027a7ee3-a371-4302-936c-dd3a282705cf","Type":"ContainerStarted","Data":"a10b936954708e58a709cb609078ce2e446b003bd4e965d7fbf366510c81ab45"} Jul 14 14:02:23.792756 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:23.792707 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" podStartSLOduration=1.7926934220000001 podStartE2EDuration="1.792693422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:02:23.792494752 +0000 UTC m=+483.019177687" watchObservedRunningTime="2026-07-14 14:02:23.792693422 +0000 UTC m=+483.019376423" Jul 14 14:02:23.931508 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:23.931472 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:24.935680 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:24.935640 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:25.762513 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:25.762484 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:25.763480 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:25.763460 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:02:27.967452 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:27.967401 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f76975456-27tr6" podUID="d4c86535-833f-40de-ad9b-c9ab2e210162" containerName="console" containerID="cri-o://b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174" gracePeriod=15 Jul 14 14:02:28.213040 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.213015 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f76975456-27tr6" Jul 14 14:02:28.296144 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296060 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-oauth-serving-cert\") pod \"d4c86535-833f-40de-ad9b-c9ab2e210162\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " Jul 14 14:02:28.296144 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296124 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-oauth-config\") pod \"d4c86535-833f-40de-ad9b-c9ab2e210162\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " Jul 14 14:02:28.296144 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296144 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-serving-cert\") pod \"d4c86535-833f-40de-ad9b-c9ab2e210162\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " Jul 14 14:02:28.296442 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296182 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-trusted-ca-bundle\") pod \"d4c86535-833f-40de-ad9b-c9ab2e210162\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " Jul 14 14:02:28.296442 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296215 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-console-config\") pod \"d4c86535-833f-40de-ad9b-c9ab2e210162\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " Jul 14 14:02:28.296442 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296374 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px4x8\" (UniqueName: \"kubernetes.io/projected/d4c86535-833f-40de-ad9b-c9ab2e210162-kube-api-access-px4x8\") pod \"d4c86535-833f-40de-ad9b-c9ab2e210162\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " Jul 14 14:02:28.296442 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296412 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-service-ca\") pod \"d4c86535-833f-40de-ad9b-c9ab2e210162\" (UID: \"d4c86535-833f-40de-ad9b-c9ab2e210162\") " Jul 14 14:02:28.296635 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296478 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d4c86535-833f-40de-ad9b-c9ab2e210162" (UID: "d4c86535-833f-40de-ad9b-c9ab2e210162"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:02:28.296736 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296716 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-oauth-serving-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:02:28.296792 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296769 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d4c86535-833f-40de-ad9b-c9ab2e210162" (UID: "d4c86535-833f-40de-ad9b-c9ab2e210162"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:02:28.296924 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296893 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-console-config" (OuterVolumeSpecName: "console-config") pod "d4c86535-833f-40de-ad9b-c9ab2e210162" (UID: "d4c86535-833f-40de-ad9b-c9ab2e210162"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:02:28.296924 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.296887 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-service-ca" (OuterVolumeSpecName: "service-ca") pod "d4c86535-833f-40de-ad9b-c9ab2e210162" (UID: "d4c86535-833f-40de-ad9b-c9ab2e210162"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:02:28.298359 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.298337 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d4c86535-833f-40de-ad9b-c9ab2e210162" (UID: "d4c86535-833f-40de-ad9b-c9ab2e210162"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 14:02:28.298458 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.298385 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c86535-833f-40de-ad9b-c9ab2e210162-kube-api-access-px4x8" (OuterVolumeSpecName: "kube-api-access-px4x8") pod "d4c86535-833f-40de-ad9b-c9ab2e210162" (UID: "d4c86535-833f-40de-ad9b-c9ab2e210162"). InnerVolumeSpecName "kube-api-access-px4x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:02:28.298503 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.298468 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d4c86535-833f-40de-ad9b-c9ab2e210162" (UID: "d4c86535-833f-40de-ad9b-c9ab2e210162"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 14:02:28.398173 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.398133 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px4x8\" (UniqueName: \"kubernetes.io/projected/d4c86535-833f-40de-ad9b-c9ab2e210162-kube-api-access-px4x8\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:02:28.398173 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.398167 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:02:28.398173 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.398179 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-oauth-config\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:02:28.398453 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.398191 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4c86535-833f-40de-ad9b-c9ab2e210162-console-serving-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:02:28.398453 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.398203 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-trusted-ca-bundle\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:02:28.398453 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.398213 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4c86535-833f-40de-ad9b-c9ab2e210162-console-config\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:02:28.778380 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.778346 2564 generic.go:356] "Generic (PLEG): container finished" podID="d4c86535-833f-40de-ad9b-c9ab2e210162" containerID="b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174" exitCode=0 Jul 14 14:02:28.778544 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.778389 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f76975456-27tr6" Jul 14 14:02:28.778544 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.778431 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f76975456-27tr6" event={"ID":"d4c86535-833f-40de-ad9b-c9ab2e210162","Type":"ContainerDied","Data":"b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174"} Jul 14 14:02:28.778544 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.778481 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f76975456-27tr6" event={"ID":"d4c86535-833f-40de-ad9b-c9ab2e210162","Type":"ContainerDied","Data":"fb2785a6358034713559e92e0718da8f43398867c3043aa96b121dc32cee521a"} Jul 14 14:02:28.778544 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.778504 2564 scope.go:117] "RemoveContainer" containerID="b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174" Jul 14 14:02:28.788098 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.788081 2564 scope.go:117] "RemoveContainer" containerID="b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174" Jul 14 14:02:28.788375 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:02:28.788352 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174\": container with ID starting with b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174 not found: ID does not exist" containerID="b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174" Jul 14 14:02:28.788466 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.788380 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174"} err="failed to get container status \"b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174\": rpc error: code = NotFound desc = could not find container \"b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174\": container with ID starting with b493d2bb1b09bcd89a4f090af333f1b337c2219d6ae38df4c3b01b9f526c2174 not found: ID does not exist" Jul 14 14:02:28.803888 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.803866 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f76975456-27tr6"] Jul 14 14:02:28.814174 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:28.814153 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f76975456-27tr6"] Jul 14 14:02:29.370099 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:02:29.370065 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c86535-833f-40de-ad9b-c9ab2e210162" path="/var/lib/kubelet/pods/d4c86535-833f-40de-ad9b-c9ab2e210162/volumes" Jul 14 14:03:54.746119 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.746087 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-kjrng"] Jul 14 14:03:54.746628 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.746477 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c86535-833f-40de-ad9b-c9ab2e210162" containerName="console" Jul 14 14:03:54.746628 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.746493 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c86535-833f-40de-ad9b-c9ab2e210162" containerName="console" Jul 14 14:03:54.746628 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.746560 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c86535-833f-40de-ad9b-c9ab2e210162" containerName="console" Jul 14 14:03:54.749460 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.749441 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:54.753890 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.753870 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sxp5g\"" Jul 14 14:03:54.754008 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.753937 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 14 14:03:54.779791 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.779764 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-kjrng"] Jul 14 14:03:54.846705 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.846670 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sq6d\" (UniqueName: \"kubernetes.io/projected/809da307-eed7-4f8b-9898-59f57b52cbb7-kube-api-access-8sq6d\") pod \"payload-processing-f9b8d9b9b-kjrng\" (UID: \"809da307-eed7-4f8b-9898-59f57b52cbb7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:54.846871 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.846770 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/809da307-eed7-4f8b-9898-59f57b52cbb7-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-kjrng\" (UID: \"809da307-eed7-4f8b-9898-59f57b52cbb7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:54.948066 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.948034 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sq6d\" (UniqueName: \"kubernetes.io/projected/809da307-eed7-4f8b-9898-59f57b52cbb7-kube-api-access-8sq6d\") pod \"payload-processing-f9b8d9b9b-kjrng\" (UID: \"809da307-eed7-4f8b-9898-59f57b52cbb7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:54.948234 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.948095 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/809da307-eed7-4f8b-9898-59f57b52cbb7-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-kjrng\" (UID: \"809da307-eed7-4f8b-9898-59f57b52cbb7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:54.948734 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.948716 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/809da307-eed7-4f8b-9898-59f57b52cbb7-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-kjrng\" (UID: \"809da307-eed7-4f8b-9898-59f57b52cbb7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:54.964857 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:54.964824 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sq6d\" (UniqueName: \"kubernetes.io/projected/809da307-eed7-4f8b-9898-59f57b52cbb7-kube-api-access-8sq6d\") pod \"payload-processing-f9b8d9b9b-kjrng\" (UID: \"809da307-eed7-4f8b-9898-59f57b52cbb7\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:55.059358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:55.059330 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:55.195939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:55.195916 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-kjrng"] Jul 14 14:03:55.197978 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:03:55.197942 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809da307_eed7_4f8b_9898_59f57b52cbb7.slice/crio-cb7299135c32168b0081037ae3d94f4757c6edc4844c7ccbf5cf5f19ef135848 WatchSource:0}: Error finding container cb7299135c32168b0081037ae3d94f4757c6edc4844c7ccbf5cf5f19ef135848: Status 404 returned error can't find the container with id cb7299135c32168b0081037ae3d94f4757c6edc4844c7ccbf5cf5f19ef135848 Jul 14 14:03:56.208353 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:56.208312 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" event={"ID":"809da307-eed7-4f8b-9898-59f57b52cbb7","Type":"ContainerStarted","Data":"cb7299135c32168b0081037ae3d94f4757c6edc4844c7ccbf5cf5f19ef135848"} Jul 14 14:03:57.220541 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:57.220500 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" event={"ID":"809da307-eed7-4f8b-9898-59f57b52cbb7","Type":"ContainerStarted","Data":"a7ae0e6aa27b4af5be723ff7103919aafc7a7154c0fcba0e6f7c170dcbc16a74"} Jul 14 14:03:57.220949 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:57.220602 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:03:57.244512 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:03:57.244437 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" podStartSLOduration=1.847333697 podStartE2EDuration="3.24441763s" totalImagesPullingTime="1.397083933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:03:57.240856461 +0000 UTC m=+576.467539379" watchObservedRunningTime="2026-07-14 14:03:57.24441763 +0000 UTC m=+576.471100547" Jul 14 14:04:03.257488 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:03.257455 2564 generic.go:356] "Generic (PLEG): container finished" podID="027a7ee3-a371-4302-936c-dd3a282705cf" containerID="b3507b0327560ec22fa2cd1830c2d5bde2f4463a91effd1992af983b873e6c01" exitCode=0 Jul 14 14:04:03.257945 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:03.257511 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" event={"ID":"027a7ee3-a371-4302-936c-dd3a282705cf","Type":"ContainerDied","Data":"b3507b0327560ec22fa2cd1830c2d5bde2f4463a91effd1992af983b873e6c01"} Jul 14 14:04:03.257945 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:03.257855 2564 scope.go:117] "RemoveContainer" containerID="b3507b0327560ec22fa2cd1830c2d5bde2f4463a91effd1992af983b873e6c01" Jul 14 14:04:03.258851 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:03.258822 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:04:03.258955 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:03.258888 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:04:03.258955 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:03.258917 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:04:04.268199 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:04.268157 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" event={"ID":"027a7ee3-a371-4302-936c-dd3a282705cf","Type":"ContainerStarted","Data":"86066af77e224e4d62bf3ac56daeec6acfb39036d96831ff1f93cc738e6ab860"} Jul 14 14:04:04.930938 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:04.930909 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:04:04.931241 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:04.931131 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4758f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 14 14:04:04.931445 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:04.931407 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" podUID="027a7ee3-a371-4302-936c-dd3a282705cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 14 14:04:05.931105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:05.931067 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4758f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 14 14:04:05.931703 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:05.931669 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" podUID="027a7ee3-a371-4302-936c-dd3a282705cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 14 14:04:06.931731 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:06.931693 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4758f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 14 14:04:06.932334 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:06.932297 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" podUID="027a7ee3-a371-4302-936c-dd3a282705cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 14 14:04:07.932128 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:07.931688 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4758f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 14 14:04:07.932128 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:07.931747 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" podUID="027a7ee3-a371-4302-936c-dd3a282705cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 14 14:04:08.232551 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:08.232454 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-kjrng" Jul 14 14:04:08.934544 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:08.934515 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:04:08.934940 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:08.934772 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:04:08.935501 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:08.935486 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4758f" Jul 14 14:04:21.255198 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.255167 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:04:21.256168 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.256149 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:04:21.792103 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.792073 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s"] Jul 14 14:04:21.794478 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.794459 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.798434 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.798414 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6dw7s\"" Jul 14 14:04:21.798551 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.798438 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 14:04:21.798551 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.798506 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 14:04:21.798551 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.798544 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 14 14:04:21.807897 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.807876 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s"] Jul 14 14:04:21.887837 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.887811 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.887953 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.887841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ncxz\" (UniqueName: \"kubernetes.io/projected/730b5115-30a9-42c0-8524-df71f30ff430-kube-api-access-4ncxz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.887953 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.887865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/730b5115-30a9-42c0-8524-df71f30ff430-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.888030 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.887974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.888030 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.888008 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.888102 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.888033 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.988910 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.988873 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/730b5115-30a9-42c0-8524-df71f30ff430-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.989079 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.989037 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.989201 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.989088 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.989201 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.989117 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.989201 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.989197 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.989406 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.989222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ncxz\" (UniqueName: \"kubernetes.io/projected/730b5115-30a9-42c0-8524-df71f30ff430-kube-api-access-4ncxz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.989498 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.989479 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.989555 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.989526 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.989593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.989556 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.991339 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.991316 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/730b5115-30a9-42c0-8524-df71f30ff430-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.991439 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.991394 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/730b5115-30a9-42c0-8524-df71f30ff430-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:21.997260 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:21.997242 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ncxz\" (UniqueName: \"kubernetes.io/projected/730b5115-30a9-42c0-8524-df71f30ff430-kube-api-access-4ncxz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s\" (UID: \"730b5115-30a9-42c0-8524-df71f30ff430\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:22.105198 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:22.105165 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:22.237349 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:22.237323 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s"] Jul 14 14:04:22.239461 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:04:22.239428 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730b5115_30a9_42c0_8524_df71f30ff430.slice/crio-1a3a39715c09f41adaad4b2a0c9535e7a4dd2db2cf9eecc5e4262b6b14931d76 WatchSource:0}: Error finding container 1a3a39715c09f41adaad4b2a0c9535e7a4dd2db2cf9eecc5e4262b6b14931d76: Status 404 returned error can't find the container with id 1a3a39715c09f41adaad4b2a0c9535e7a4dd2db2cf9eecc5e4262b6b14931d76 Jul 14 14:04:22.355151 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:22.355118 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" event={"ID":"730b5115-30a9-42c0-8524-df71f30ff430","Type":"ContainerStarted","Data":"1a3a39715c09f41adaad4b2a0c9535e7a4dd2db2cf9eecc5e4262b6b14931d76"} Jul 14 14:04:24.367142 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:24.367101 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" event={"ID":"730b5115-30a9-42c0-8524-df71f30ff430","Type":"ContainerStarted","Data":"048b2e728ac923ebfe58c9c3f56865591faed421f805f1b11464ff9319367757"} Jul 14 14:04:24.386165 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:24.386113 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" podStartSLOduration=2.176216879 podStartE2EDuration="3.386098918s" totalImagesPullingTime="1.209882039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:04:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:04:24.385782329 +0000 UTC m=+603.612465265" watchObservedRunningTime="2026-07-14 14:04:24.386098918 +0000 UTC m=+603.612781834" Jul 14 14:04:32.105389 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:32.105338 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:32.105937 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:32.105418 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:32.118021 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:32.117999 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:04:32.418150 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:04:32.418076 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s" Jul 14 14:05:20.989869 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:20.989828 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk"] Jul 14 14:05:20.993407 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:20.993385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:20.996178 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:20.996160 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 14:05:21.004117 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.004096 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk"] Jul 14 14:05:21.090951 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.090916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.091129 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.090957 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.091129 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.090988 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bf0d3c9-0272-4953-9671-caa924b0cf28-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.091129 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.091027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pktzv\" (UniqueName: \"kubernetes.io/projected/3bf0d3c9-0272-4953-9671-caa924b0cf28-kube-api-access-pktzv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.091129 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.091050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.091316 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.091155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.192482 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.192482 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192489 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.192734 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bf0d3c9-0272-4953-9671-caa924b0cf28-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.192734 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pktzv\" (UniqueName: \"kubernetes.io/projected/3bf0d3c9-0272-4953-9671-caa924b0cf28-kube-api-access-pktzv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.192734 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192547 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.192734 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192582 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.192910 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192881 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.192969 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.193006 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.192986 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.194853 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.194825 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3bf0d3c9-0272-4953-9671-caa924b0cf28-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.195756 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.195738 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 14:05:21.201836 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.201817 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pktzv\" (UniqueName: \"kubernetes.io/projected/3bf0d3c9-0272-4953-9671-caa924b0cf28-kube-api-access-pktzv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.205354 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.205333 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bf0d3c9-0272-4953-9671-caa924b0cf28-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk\" (UID: \"3bf0d3c9-0272-4953-9671-caa924b0cf28\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.304776 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.304752 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:21.439691 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.439661 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk"] Jul 14 14:05:21.440396 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:05:21.440375 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf0d3c9_0272_4953_9671_caa924b0cf28.slice/crio-b61bcad1514e06eae9c291d51501d4f7c53201c552a63e310741003a70fb6492 WatchSource:0}: Error finding container b61bcad1514e06eae9c291d51501d4f7c53201c552a63e310741003a70fb6492: Status 404 returned error can't find the container with id b61bcad1514e06eae9c291d51501d4f7c53201c552a63e310741003a70fb6492 Jul 14 14:05:21.441602 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.441584 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 14:05:21.651752 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:21.651725 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" event={"ID":"3bf0d3c9-0272-4953-9671-caa924b0cf28","Type":"ContainerStarted","Data":"b61bcad1514e06eae9c291d51501d4f7c53201c552a63e310741003a70fb6492"} Jul 14 14:05:22.658603 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:22.658566 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" event={"ID":"3bf0d3c9-0272-4953-9671-caa924b0cf28","Type":"ContainerStarted","Data":"07e42ee2f655235be3587b7ac1a1b5e3f5e61503a3ad0cf9e017f66d0f6d869d"} Jul 14 14:05:22.678320 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:22.678257 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" podStartSLOduration=2.472146509 podStartE2EDuration="2.678239195s" totalImagesPullingTime="206.092686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:05:22.675683862 +0000 UTC m=+661.902366799" watchObservedRunningTime="2026-07-14 14:05:22.678239195 +0000 UTC m=+661.904922111" Jul 14 14:05:31.305942 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:31.305905 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:31.306515 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:31.305978 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:31.318355 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:31.318330 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:05:31.717438 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:05:31.717358 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk" Jul 14 14:09:21.290140 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:09:21.290108 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:09:21.290637 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:09:21.290229 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:14:21.322972 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:14:21.322935 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:14:21.325488 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:14:21.324942 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:15:00.143440 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:00.143404 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992"] Jul 14 14:15:00.146710 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:00.146693 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992" Jul 14 14:15:00.149251 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:00.149230 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nk9bs\"" Jul 14 14:15:00.154556 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:00.154333 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992"] Jul 14 14:15:00.157747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:00.157721 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992" Jul 14 14:15:00.294295 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:00.294248 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992"] Jul 14 14:15:00.295936 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:15:00.295905 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31c8d153_2908_4f5a_9e50_3a9d98738d86.slice/crio-1990a2e65b7b32e9f3ffe6f95480e590d2f94401b8d5e7cddb6c680d16aff9e4 WatchSource:0}: Error finding container 1990a2e65b7b32e9f3ffe6f95480e590d2f94401b8d5e7cddb6c680d16aff9e4: Status 404 returned error can't find the container with id 1990a2e65b7b32e9f3ffe6f95480e590d2f94401b8d5e7cddb6c680d16aff9e4 Jul 14 14:15:00.297845 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:00.297826 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 14:15:00.557139 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:00.557112 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992" event={"ID":"31c8d153-2908-4f5a-9e50-3a9d98738d86","Type":"ContainerStarted","Data":"1990a2e65b7b32e9f3ffe6f95480e590d2f94401b8d5e7cddb6c680d16aff9e4"} Jul 14 14:15:03.576104 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:03.576071 2564 generic.go:356] "Generic (PLEG): container finished" podID="31c8d153-2908-4f5a-9e50-3a9d98738d86" containerID="6d5fd051c840cbed43e42ebd7cff2b88f13e0db95725e4ccd9feb3fc58eaefa2" exitCode=0 Jul 14 14:15:03.576495 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:03.576130 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992" event={"ID":"31c8d153-2908-4f5a-9e50-3a9d98738d86","Type":"ContainerDied","Data":"6d5fd051c840cbed43e42ebd7cff2b88f13e0db95725e4ccd9feb3fc58eaefa2"} Jul 14 14:15:04.709975 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:04.709945 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992" Jul 14 14:15:05.587728 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:05.587699 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992" Jul 14 14:15:05.587892 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:05.587729 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733975-xk992" event={"ID":"31c8d153-2908-4f5a-9e50-3a9d98738d86","Type":"ContainerDied","Data":"1990a2e65b7b32e9f3ffe6f95480e590d2f94401b8d5e7cddb6c680d16aff9e4"} Jul 14 14:15:05.587892 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:15:05.587776 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1990a2e65b7b32e9f3ffe6f95480e590d2f94401b8d5e7cddb6c680d16aff9e4" Jul 14 14:19:21.356852 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:21.356725 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:19:21.359811 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:21.359719 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:19:50.597519 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.597437 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5"] Jul 14 14:19:50.597863 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.597785 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31c8d153-2908-4f5a-9e50-3a9d98738d86" containerName="cleanup" Jul 14 14:19:50.597863 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.597795 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="31c8d153-2908-4f5a-9e50-3a9d98738d86" containerName="cleanup" Jul 14 14:19:50.597863 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.597857 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="31c8d153-2908-4f5a-9e50-3a9d98738d86" containerName="cleanup" Jul 14 14:19:50.600993 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.600975 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.603658 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.603635 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4f7140c9-openshift-default-dockercfg-jc7f8\"" Jul 14 14:19:50.624422 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.624391 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5"] Jul 14 14:19:50.718392 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718358 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-token\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.718588 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4r9s\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-kube-api-access-w4r9s\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.718588 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718457 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-certs\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.718588 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-podinfo\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.718588 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718528 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istiod-ca-cert\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.718588 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-envoy\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.718844 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718598 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-credential-socket\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.718844 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718693 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-data\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.718844 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.718751 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-socket\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.819754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.819713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-token\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.819936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.819783 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4r9s\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-kube-api-access-w4r9s\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.819936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.819808 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-certs\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.819936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.819830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-podinfo\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.819936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.819849 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istiod-ca-cert\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.819936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.819877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-envoy\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.820189 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.820004 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-credential-socket\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.820189 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.820131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-data\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.820328 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.820193 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-socket\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.820328 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.820226 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-certs\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.820465 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.820438 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-credential-socket\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.820529 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.820514 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-data\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.820602 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.820582 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-socket\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.820736 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.820713 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istiod-ca-cert\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.822125 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.822105 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-envoy\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.822373 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.822352 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-podinfo\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.827914 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.827890 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4r9s\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-kube-api-access-w4r9s\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.827914 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.827912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-token\") pod \"e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:50.911979 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:50.911898 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:51.057131 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:51.057108 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5"] Jul 14 14:19:51.058955 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:19:51.058923 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode20952eb_f2a3_48fb_95cd_7025cac1a9fc.slice/crio-96fac1c6180d534bec371db4d4096f78ed73b701812465d5149ec5652afdaeb1 WatchSource:0}: Error finding container 96fac1c6180d534bec371db4d4096f78ed73b701812465d5149ec5652afdaeb1: Status 404 returned error can't find the container with id 96fac1c6180d534bec371db4d4096f78ed73b701812465d5149ec5652afdaeb1 Jul 14 14:19:51.060986 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:51.060941 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:19:51.061088 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:51.061029 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:19:51.061088 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:51.061076 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:19:52.049704 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:52.049668 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" event={"ID":"e20952eb-f2a3-48fb-95cd-7025cac1a9fc","Type":"ContainerStarted","Data":"7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee"} Jul 14 14:19:52.050157 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:52.049749 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" event={"ID":"e20952eb-f2a3-48fb-95cd-7025cac1a9fc","Type":"ContainerStarted","Data":"96fac1c6180d534bec371db4d4096f78ed73b701812465d5149ec5652afdaeb1"} Jul 14 14:19:52.074821 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:52.074775 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" podStartSLOduration=2.0747603039999998 podStartE2EDuration="2.074760304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:19:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:19:52.071937365 +0000 UTC m=+1531.298620301" watchObservedRunningTime="2026-07-14 14:19:52.074760304 +0000 UTC m=+1531.301443222" Jul 14 14:19:52.913113 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:52.913080 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:52.917728 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:52.917702 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:53.055146 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:53.055118 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:53.056065 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:53.056043 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:19:57.006053 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.006017 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf"] Jul 14 14:19:57.009785 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.009769 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" Jul 14 14:19:57.012661 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.012640 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:19:57.012661 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.012660 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nk9bs\"" Jul 14 14:19:57.017853 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.017760 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf"] Jul 14 14:19:57.178464 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.178422 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782ec21-a779-40b2-88b5-90f907bb3ae7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf\" (UID: \"4782ec21-a779-40b2-88b5-90f907bb3ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" Jul 14 14:19:57.279358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.279250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782ec21-a779-40b2-88b5-90f907bb3ae7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf\" (UID: \"4782ec21-a779-40b2-88b5-90f907bb3ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" Jul 14 14:19:57.279931 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.279907 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782ec21-a779-40b2-88b5-90f907bb3ae7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf\" (UID: \"4782ec21-a779-40b2-88b5-90f907bb3ae7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" Jul 14 14:19:57.322991 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.322943 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" Jul 14 14:19:57.503589 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:57.503555 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf"] Jul 14 14:19:57.506859 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:19:57.506823 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4782ec21_a779_40b2_88b5_90f907bb3ae7.slice/crio-04d74048cfea8e1895728a2b2c652bbe7ac550a4128b15cd509f46ae122d953c WatchSource:0}: Error finding container 04d74048cfea8e1895728a2b2c652bbe7ac550a4128b15cd509f46ae122d953c: Status 404 returned error can't find the container with id 04d74048cfea8e1895728a2b2c652bbe7ac550a4128b15cd509f46ae122d953c Jul 14 14:19:58.082269 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:58.082228 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" event={"ID":"4782ec21-a779-40b2-88b5-90f907bb3ae7","Type":"ContainerStarted","Data":"fa98be5d6f422ecd5e7022748cac3064bbc2f22a758419e384752aac4c122bf1"} Jul 14 14:19:58.082779 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:58.082312 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" event={"ID":"4782ec21-a779-40b2-88b5-90f907bb3ae7","Type":"ContainerStarted","Data":"04d74048cfea8e1895728a2b2c652bbe7ac550a4128b15cd509f46ae122d953c"} Jul 14 14:19:58.099229 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:58.099186 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" podStartSLOduration=2.099171644 podStartE2EDuration="2.099171644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:19:58.097884662 +0000 UTC m=+1537.324567604" watchObservedRunningTime="2026-07-14 14:19:58.099171644 +0000 UTC m=+1537.325854561" Jul 14 14:19:59.088890 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:59.088852 2564 generic.go:356] "Generic (PLEG): container finished" podID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerID="fa98be5d6f422ecd5e7022748cac3064bbc2f22a758419e384752aac4c122bf1" exitCode=7 Jul 14 14:19:59.089367 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:59.088939 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" event={"ID":"4782ec21-a779-40b2-88b5-90f907bb3ae7","Type":"ContainerDied","Data":"fa98be5d6f422ecd5e7022748cac3064bbc2f22a758419e384752aac4c122bf1"} Jul 14 14:19:59.089367 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:19:59.089137 2564 scope.go:117] "RemoveContainer" containerID="fa98be5d6f422ecd5e7022748cac3064bbc2f22a758419e384752aac4c122bf1" Jul 14 14:20:00.096646 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:00.096612 2564 generic.go:356] "Generic (PLEG): container finished" podID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerID="e2351d2c4501bec8b13a00cefbe5b45cdc6f7f98500e1231795eef6806572830" exitCode=7 Jul 14 14:20:00.097153 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:00.096692 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" event={"ID":"4782ec21-a779-40b2-88b5-90f907bb3ae7","Type":"ContainerDied","Data":"e2351d2c4501bec8b13a00cefbe5b45cdc6f7f98500e1231795eef6806572830"} Jul 14 14:20:00.097153 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:00.096754 2564 scope.go:117] "RemoveContainer" containerID="fa98be5d6f422ecd5e7022748cac3064bbc2f22a758419e384752aac4c122bf1" Jul 14 14:20:00.097153 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:00.096925 2564 scope.go:117] "RemoveContainer" containerID="e2351d2c4501bec8b13a00cefbe5b45cdc6f7f98500e1231795eef6806572830" Jul 14 14:20:00.097335 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:20:00.097193 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf_odh-ai-gateway-infra(4782ec21-a779-40b2-88b5-90f907bb3ae7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" Jul 14 14:20:01.103847 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:01.103822 2564 scope.go:117] "RemoveContainer" containerID="e2351d2c4501bec8b13a00cefbe5b45cdc6f7f98500e1231795eef6806572830" Jul 14 14:20:01.104206 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:20:01.104024 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf_odh-ai-gateway-infra(4782ec21-a779-40b2-88b5-90f907bb3ae7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" Jul 14 14:20:03.116368 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:03.116331 2564 generic.go:356] "Generic (PLEG): container finished" podID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerID="7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee" exitCode=0 Jul 14 14:20:03.116768 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:03.116401 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" event={"ID":"e20952eb-f2a3-48fb-95cd-7025cac1a9fc","Type":"ContainerDied","Data":"7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee"} Jul 14 14:20:03.116809 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:03.116783 2564 scope.go:117] "RemoveContainer" containerID="7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee" Jul 14 14:20:03.117742 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:03.117716 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:20:03.117798 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:03.117785 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:20:03.117839 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:03.117821 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:20:04.122856 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:04.122822 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" event={"ID":"e20952eb-f2a3-48fb-95cd-7025cac1a9fc","Type":"ContainerStarted","Data":"320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08"} Jul 14 14:20:04.912221 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:04.912192 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:20:04.913322 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:04.913299 2564 patch_prober.go:28] interesting pod/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 14 14:20:04.913382 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:04.913348 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 14 14:20:05.912672 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:05.912636 2564 patch_prober.go:28] interesting pod/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 14 14:20:05.913067 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:05.912692 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 14 14:20:06.912496 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:06.912460 2564 patch_prober.go:28] interesting pod/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 14 14:20:06.912712 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:06.912530 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 14 14:20:07.912620 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:07.912583 2564 patch_prober.go:28] interesting pod/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 14 14:20:07.912797 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:07.912636 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 14 14:20:08.055623 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:08.055588 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:20:08.916770 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:08.916741 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:20:08.917775 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:08.917758 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:20:12.364307 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:12.364256 2564 scope.go:117] "RemoveContainer" containerID="e2351d2c4501bec8b13a00cefbe5b45cdc6f7f98500e1231795eef6806572830" Jul 14 14:20:13.184409 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:13.184379 2564 generic.go:356] "Generic (PLEG): container finished" podID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerID="dce1483ba7371a64d17b94aa8f7cc0bbe5e1036e462c79c5f6702da462fb924b" exitCode=0 Jul 14 14:20:13.184584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:13.184455 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" event={"ID":"4782ec21-a779-40b2-88b5-90f907bb3ae7","Type":"ContainerDied","Data":"dce1483ba7371a64d17b94aa8f7cc0bbe5e1036e462c79c5f6702da462fb924b"} Jul 14 14:20:13.184584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:13.184525 2564 scope.go:117] "RemoveContainer" containerID="e2351d2c4501bec8b13a00cefbe5b45cdc6f7f98500e1231795eef6806572830" Jul 14 14:20:14.231503 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:14.231472 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf"] Jul 14 14:20:14.330450 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:14.330428 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" Jul 14 14:20:14.428875 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:14.428846 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782ec21-a779-40b2-88b5-90f907bb3ae7-maas-api-service-ca\") pod \"4782ec21-a779-40b2-88b5-90f907bb3ae7\" (UID: \"4782ec21-a779-40b2-88b5-90f907bb3ae7\") " Jul 14 14:20:14.429301 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:14.429252 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4782ec21-a779-40b2-88b5-90f907bb3ae7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4782ec21-a779-40b2-88b5-90f907bb3ae7" (UID: "4782ec21-a779-40b2-88b5-90f907bb3ae7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:20:14.429424 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:14.429393 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4782ec21-a779-40b2-88b5-90f907bb3ae7-maas-api-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:20:15.199961 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:15.199928 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" event={"ID":"4782ec21-a779-40b2-88b5-90f907bb3ae7","Type":"ContainerDied","Data":"04d74048cfea8e1895728a2b2c652bbe7ac550a4128b15cd509f46ae122d953c"} Jul 14 14:20:15.199961 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:15.199948 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf" Jul 14 14:20:15.200164 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:15.199976 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04d74048cfea8e1895728a2b2c652bbe7ac550a4128b15cd509f46ae122d953c" Jul 14 14:20:15.229781 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:15.229753 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf"] Jul 14 14:20:15.235602 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:15.235581 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-c2vlf"] Jul 14 14:20:15.370507 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:15.370479 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" path="/var/lib/kubelet/pods/4782ec21-a779-40b2-88b5-90f907bb3ae7/volumes" Jul 14 14:20:47.071622 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.071580 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb"] Jul 14 14:20:47.072212 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.072174 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:20:47.072212 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.072194 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:20:47.072212 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.072206 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:20:47.072212 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.072214 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:20:47.072449 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.072232 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:20:47.072449 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.072240 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:20:47.072449 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.072344 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:20:47.072449 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.072359 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:20:47.074668 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.074645 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" Jul 14 14:20:47.077922 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.077902 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:20:47.078350 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.078332 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nk9bs\"" Jul 14 14:20:47.089390 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.089360 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb"] Jul 14 14:20:47.238468 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.238434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3add4b-95a5-48ac-97de-483734dfd3ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb\" (UID: \"5f3add4b-95a5-48ac-97de-483734dfd3ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" Jul 14 14:20:47.339966 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.339887 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3add4b-95a5-48ac-97de-483734dfd3ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb\" (UID: \"5f3add4b-95a5-48ac-97de-483734dfd3ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" Jul 14 14:20:47.340524 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.340505 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3add4b-95a5-48ac-97de-483734dfd3ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb\" (UID: \"5f3add4b-95a5-48ac-97de-483734dfd3ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" Jul 14 14:20:47.384435 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.384407 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" Jul 14 14:20:47.514785 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:47.514755 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb"] Jul 14 14:20:47.517548 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:20:47.517518 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f3add4b_95a5_48ac_97de_483734dfd3ac.slice/crio-6ae6a1aef9cbc73d3ecbef6e1395c33143a3b56e37fdd61575b61088e5cf2db6 WatchSource:0}: Error finding container 6ae6a1aef9cbc73d3ecbef6e1395c33143a3b56e37fdd61575b61088e5cf2db6: Status 404 returned error can't find the container with id 6ae6a1aef9cbc73d3ecbef6e1395c33143a3b56e37fdd61575b61088e5cf2db6 Jul 14 14:20:48.372188 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:48.372151 2564 generic.go:356] "Generic (PLEG): container finished" podID="5f3add4b-95a5-48ac-97de-483734dfd3ac" containerID="28c18fa9e30d35b25095259d6d5a61dfd6dcfc2b96fabb0ca8815f84f737987b" exitCode=0 Jul 14 14:20:48.372594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:48.372232 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" event={"ID":"5f3add4b-95a5-48ac-97de-483734dfd3ac","Type":"ContainerDied","Data":"28c18fa9e30d35b25095259d6d5a61dfd6dcfc2b96fabb0ca8815f84f737987b"} Jul 14 14:20:48.372594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:48.372275 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" event={"ID":"5f3add4b-95a5-48ac-97de-483734dfd3ac","Type":"ContainerStarted","Data":"6ae6a1aef9cbc73d3ecbef6e1395c33143a3b56e37fdd61575b61088e5cf2db6"} Jul 14 14:20:49.508012 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:49.507989 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" Jul 14 14:20:49.556844 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:49.556810 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3add4b-95a5-48ac-97de-483734dfd3ac-maas-api-service-ca\") pod \"5f3add4b-95a5-48ac-97de-483734dfd3ac\" (UID: \"5f3add4b-95a5-48ac-97de-483734dfd3ac\") " Jul 14 14:20:49.557178 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:49.557156 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f3add4b-95a5-48ac-97de-483734dfd3ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f3add4b-95a5-48ac-97de-483734dfd3ac" (UID: "5f3add4b-95a5-48ac-97de-483734dfd3ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:20:49.657608 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:49.657518 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f3add4b-95a5-48ac-97de-483734dfd3ac-maas-api-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:20:50.383783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:50.383757 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" Jul 14 14:20:50.383960 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:50.383784 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb" event={"ID":"5f3add4b-95a5-48ac-97de-483734dfd3ac","Type":"ContainerDied","Data":"6ae6a1aef9cbc73d3ecbef6e1395c33143a3b56e37fdd61575b61088e5cf2db6"} Jul 14 14:20:50.383960 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:20:50.383825 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ae6a1aef9cbc73d3ecbef6e1395c33143a3b56e37fdd61575b61088e5cf2db6" Jul 14 14:21:12.612328 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:12.612228 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5"] Jul 14 14:21:12.612813 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:12.612469 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" containerID="cri-o://320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08" gracePeriod=30 Jul 14 14:21:13.854593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.854563 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77"] Jul 14 14:21:13.854940 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.854930 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f3add4b-95a5-48ac-97de-483734dfd3ac" containerName="revoke-keys" Jul 14 14:21:13.854980 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.854941 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f3add4b-95a5-48ac-97de-483734dfd3ac" containerName="revoke-keys" Jul 14 14:21:13.855013 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.854995 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f3add4b-95a5-48ac-97de-483734dfd3ac" containerName="revoke-keys" Jul 14 14:21:13.855013 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.855007 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4782ec21-a779-40b2-88b5-90f907bb3ae7" containerName="revoke-keys" Jul 14 14:21:13.857091 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.857076 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.860577 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.860550 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-26a3f6d9-gw-openshift-default-dockercfg-smxdh\"" Jul 14 14:21:13.880596 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.880564 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77"] Jul 14 14:21:13.979488 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-socket\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.979488 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-data\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.979777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979514 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba62348-0341-4772-b2b8-f0c56f10b69e-istiod-ca-cert\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.979777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979576 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-certs\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.979777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979624 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-credential-socket\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.979777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979648 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-podinfo\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.979777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979676 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-token\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.979777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-envoy\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:13.979777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:13.979744 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgbsn\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-kube-api-access-bgbsn\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.080757 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.080713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-certs\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.080757 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.080769 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-credential-socket\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081029 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.080785 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-podinfo\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081029 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.080893 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-token\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081134 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081056 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-envoy\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081134 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgbsn\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-kube-api-access-bgbsn\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081227 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081171 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-credential-socket\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081227 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081214 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-certs\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081375 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081239 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-socket\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081375 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081266 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-data\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081375 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba62348-0341-4772-b2b8-f0c56f10b69e-istiod-ca-cert\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081588 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081567 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-socket\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081647 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081602 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-data\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.081914 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.081897 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba62348-0341-4772-b2b8-f0c56f10b69e-istiod-ca-cert\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.083764 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.083734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-envoy\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.083952 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.083936 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-podinfo\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.089867 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.089842 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-token\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.090090 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.090073 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgbsn\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-kube-api-access-bgbsn\") pod \"e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.168405 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.168312 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:14.304470 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:21:14.304437 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba62348_0341_4772_b2b8_f0c56f10b69e.slice/crio-814fea111f7a9425abfdf67e22fb7a61b124a58bf74d48a04c5a8395ab5962b3 WatchSource:0}: Error finding container 814fea111f7a9425abfdf67e22fb7a61b124a58bf74d48a04c5a8395ab5962b3: Status 404 returned error can't find the container with id 814fea111f7a9425abfdf67e22fb7a61b124a58bf74d48a04c5a8395ab5962b3 Jul 14 14:21:14.306207 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.306178 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77"] Jul 14 14:21:14.306467 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.306436 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:21:14.306522 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.306499 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:21:14.306558 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.306525 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:21:14.511391 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.511272 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" event={"ID":"eba62348-0341-4772-b2b8-f0c56f10b69e","Type":"ContainerStarted","Data":"17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f"} Jul 14 14:21:14.511391 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.511340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" event={"ID":"eba62348-0341-4772-b2b8-f0c56f10b69e","Type":"ContainerStarted","Data":"814fea111f7a9425abfdf67e22fb7a61b124a58bf74d48a04c5a8395ab5962b3"} Jul 14 14:21:14.544108 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:14.544041 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" podStartSLOduration=1.544018771 podStartE2EDuration="1.544018771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:21:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:21:14.539177028 +0000 UTC m=+1613.765859946" watchObservedRunningTime="2026-07-14 14:21:14.544018771 +0000 UTC m=+1613.770701691" Jul 14 14:21:15.168777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:15.168740 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:15.173550 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:15.173527 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:15.516487 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:15.516404 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:15.517310 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:15.517267 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:17.864220 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:17.864198 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:21:18.019404 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019361 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-credential-socket\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.019404 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019408 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-token\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.019664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019464 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4r9s\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-kube-api-access-w4r9s\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.019664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019489 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-envoy\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.019664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019515 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istiod-ca-cert\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.019664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019547 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-socket\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.019664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019582 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-certs\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.019664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019605 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-podinfo\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.019664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019626 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-data\") pod \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\" (UID: \"e20952eb-f2a3-48fb-95cd-7025cac1a9fc\") " Jul 14 14:21:18.020232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019703 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:21:18.020232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.019909 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:21:18.020232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.020006 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:21:18.020232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.020020 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.020232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.020040 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.020533 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.020246 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:21:18.020850 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.020822 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-data" (OuterVolumeSpecName: "istio-data") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:21:18.021894 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.021870 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-kube-api-access-w4r9s" (OuterVolumeSpecName: "kube-api-access-w4r9s") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "kube-api-access-w4r9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:21:18.021894 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.021886 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:21:18.022022 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.021943 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:21:18.022215 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.022202 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e20952eb-f2a3-48fb-95cd-7025cac1a9fc" (UID: "e20952eb-f2a3-48fb-95cd-7025cac1a9fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:21:18.120591 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.120556 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.120591 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.120585 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.120591 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.120595 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.120827 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.120605 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.120827 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.120615 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.120827 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.120623 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4r9s\" (UniqueName: \"kubernetes.io/projected/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-kube-api-access-w4r9s\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.120827 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.120631 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e20952eb-f2a3-48fb-95cd-7025cac1a9fc-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:18.533612 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.533579 2564 generic.go:356] "Generic (PLEG): container finished" podID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerID="320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08" exitCode=0 Jul 14 14:21:18.533812 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.533639 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" Jul 14 14:21:18.533812 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.533669 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" event={"ID":"e20952eb-f2a3-48fb-95cd-7025cac1a9fc","Type":"ContainerDied","Data":"320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08"} Jul 14 14:21:18.533812 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.533722 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5" event={"ID":"e20952eb-f2a3-48fb-95cd-7025cac1a9fc","Type":"ContainerDied","Data":"96fac1c6180d534bec371db4d4096f78ed73b701812465d5149ec5652afdaeb1"} Jul 14 14:21:18.533812 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.533745 2564 scope.go:117] "RemoveContainer" containerID="320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08" Jul 14 14:21:18.543465 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.543449 2564 scope.go:117] "RemoveContainer" containerID="7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee" Jul 14 14:21:18.551422 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.551397 2564 scope.go:117] "RemoveContainer" containerID="320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08" Jul 14 14:21:18.551682 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:21:18.551664 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08\": container with ID starting with 320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08 not found: ID does not exist" containerID="320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08" Jul 14 14:21:18.551752 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.551690 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08"} err="failed to get container status \"320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08\": rpc error: code = NotFound desc = could not find container \"320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08\": container with ID starting with 320abafa54ef0b053a7d3218c955c3c7bcb7e88a6c91b75cfd757637ec7e0d08 not found: ID does not exist" Jul 14 14:21:18.551752 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.551706 2564 scope.go:117] "RemoveContainer" containerID="7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee" Jul 14 14:21:18.551934 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:21:18.551916 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee\": container with ID starting with 7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee not found: ID does not exist" containerID="7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee" Jul 14 14:21:18.551975 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.551942 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee"} err="failed to get container status \"7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee\": rpc error: code = NotFound desc = could not find container \"7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee\": container with ID starting with 7e3718bb9b4979b3a2e45368d3e37c3068c9e50e3205558a7f7848d2742bf0ee not found: ID does not exist" Jul 14 14:21:18.557784 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.557752 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5"] Jul 14 14:21:18.561955 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:18.561936 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4f7140c9-openshift-default-66dc55444c-b9cj5"] Jul 14 14:21:19.372612 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:19.372580 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" path="/var/lib/kubelet/pods/e20952eb-f2a3-48fb-95cd-7025cac1a9fc/volumes" Jul 14 14:21:20.316064 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.316031 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r"] Jul 14 14:21:20.316417 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.316403 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" Jul 14 14:21:20.316474 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.316418 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" Jul 14 14:21:20.316521 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.316502 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" Jul 14 14:21:20.316521 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.316512 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" Jul 14 14:21:20.319539 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.319521 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" Jul 14 14:21:20.322455 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.322433 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:21:20.322847 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.322653 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nk9bs\"" Jul 14 14:21:20.335944 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.335914 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r"] Jul 14 14:21:20.450822 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.450778 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65822b17-1b87-4297-b62b-72a98e337166-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r\" (UID: \"65822b17-1b87-4297-b62b-72a98e337166\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" Jul 14 14:21:20.551495 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.551450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65822b17-1b87-4297-b62b-72a98e337166-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r\" (UID: \"65822b17-1b87-4297-b62b-72a98e337166\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" Jul 14 14:21:20.552358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.552334 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65822b17-1b87-4297-b62b-72a98e337166-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r\" (UID: \"65822b17-1b87-4297-b62b-72a98e337166\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" Jul 14 14:21:20.660269 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.660165 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" Jul 14 14:21:20.816369 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:20.816346 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r"] Jul 14 14:21:20.818040 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:21:20.818010 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65822b17_1b87_4297_b62b_72a98e337166.slice/crio-ce4f7ab880192feb98c9096a3dad955e0395317b9e0168f4b1a80875deb04dc0 WatchSource:0}: Error finding container ce4f7ab880192feb98c9096a3dad955e0395317b9e0168f4b1a80875deb04dc0: Status 404 returned error can't find the container with id ce4f7ab880192feb98c9096a3dad955e0395317b9e0168f4b1a80875deb04dc0 Jul 14 14:21:21.559559 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:21.559520 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" event={"ID":"65822b17-1b87-4297-b62b-72a98e337166","Type":"ContainerStarted","Data":"1f27ca23a880b204ba8d297cd5e1906599c8bd5da9f086f67cb6b8ba2c5226d7"} Jul 14 14:21:21.559942 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:21.559579 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" event={"ID":"65822b17-1b87-4297-b62b-72a98e337166","Type":"ContainerStarted","Data":"ce4f7ab880192feb98c9096a3dad955e0395317b9e0168f4b1a80875deb04dc0"} Jul 14 14:21:21.577351 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:21.577298 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" podStartSLOduration=1.577267309 podStartE2EDuration="1.577267309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:21:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:21:21.575693319 +0000 UTC m=+1620.802376246" watchObservedRunningTime="2026-07-14 14:21:21.577267309 +0000 UTC m=+1620.803950295" Jul 14 14:21:22.566446 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:22.566410 2564 generic.go:356] "Generic (PLEG): container finished" podID="65822b17-1b87-4297-b62b-72a98e337166" containerID="1f27ca23a880b204ba8d297cd5e1906599c8bd5da9f086f67cb6b8ba2c5226d7" exitCode=7 Jul 14 14:21:22.566818 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:22.566502 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" event={"ID":"65822b17-1b87-4297-b62b-72a98e337166","Type":"ContainerDied","Data":"1f27ca23a880b204ba8d297cd5e1906599c8bd5da9f086f67cb6b8ba2c5226d7"} Jul 14 14:21:22.566818 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:22.566638 2564 scope.go:117] "RemoveContainer" containerID="1f27ca23a880b204ba8d297cd5e1906599c8bd5da9f086f67cb6b8ba2c5226d7" Jul 14 14:21:23.573263 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:23.573229 2564 generic.go:356] "Generic (PLEG): container finished" podID="65822b17-1b87-4297-b62b-72a98e337166" containerID="54578f86e053fbe073c0bd18812d6dce27c75e7825d342d46b9cd75a1ebda534" exitCode=7 Jul 14 14:21:23.573664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:23.573318 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" event={"ID":"65822b17-1b87-4297-b62b-72a98e337166","Type":"ContainerDied","Data":"54578f86e053fbe073c0bd18812d6dce27c75e7825d342d46b9cd75a1ebda534"} Jul 14 14:21:23.573664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:23.573378 2564 scope.go:117] "RemoveContainer" containerID="1f27ca23a880b204ba8d297cd5e1906599c8bd5da9f086f67cb6b8ba2c5226d7" Jul 14 14:21:23.573664 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:23.573523 2564 scope.go:117] "RemoveContainer" containerID="54578f86e053fbe073c0bd18812d6dce27c75e7825d342d46b9cd75a1ebda534" Jul 14 14:21:23.573815 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:21:23.573775 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r_odh-ai-gateway-infra(65822b17-1b87-4297-b62b-72a98e337166)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" podUID="65822b17-1b87-4297-b62b-72a98e337166" Jul 14 14:21:24.581717 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:24.581691 2564 scope.go:117] "RemoveContainer" containerID="54578f86e053fbe073c0bd18812d6dce27c75e7825d342d46b9cd75a1ebda534" Jul 14 14:21:24.582112 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:21:24.581887 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r_odh-ai-gateway-infra(65822b17-1b87-4297-b62b-72a98e337166)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" podUID="65822b17-1b87-4297-b62b-72a98e337166" Jul 14 14:21:26.594412 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:26.594380 2564 generic.go:356] "Generic (PLEG): container finished" podID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerID="17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f" exitCode=0 Jul 14 14:21:26.594412 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:26.594416 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" event={"ID":"eba62348-0341-4772-b2b8-f0c56f10b69e","Type":"ContainerDied","Data":"17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f"} Jul 14 14:21:26.595063 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:26.594777 2564 scope.go:117] "RemoveContainer" containerID="17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f" Jul 14 14:21:26.595759 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:26.595731 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:21:26.595820 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:26.595807 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:21:26.595857 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:26.595840 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:21:27.601274 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:27.601241 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" event={"ID":"eba62348-0341-4772-b2b8-f0c56f10b69e","Type":"ContainerStarted","Data":"64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a"} Jul 14 14:21:28.169199 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:28.169156 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:28.170443 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:28.170416 2564 patch_prober.go:28] interesting pod/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 14 14:21:28.170561 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:28.170476 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 14 14:21:29.169654 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:29.169616 2564 patch_prober.go:28] interesting pod/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 14 14:21:29.170049 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:29.169676 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 14 14:21:30.169718 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:30.169681 2564 patch_prober.go:28] interesting pod/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 14 14:21:30.170103 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:30.169740 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 14 14:21:30.516646 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:30.516550 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:31.169355 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:31.169319 2564 patch_prober.go:28] interesting pod/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jul 14 14:21:31.169535 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:31.169377 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jul 14 14:21:32.172203 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:32.172168 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:32.173167 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:32.173139 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:21:35.364830 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:35.364800 2564 scope.go:117] "RemoveContainer" containerID="54578f86e053fbe073c0bd18812d6dce27c75e7825d342d46b9cd75a1ebda534" Jul 14 14:21:35.644751 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:35.644722 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" event={"ID":"65822b17-1b87-4297-b62b-72a98e337166","Type":"ContainerStarted","Data":"002739b70f8b802ec352a6944df9c15c85f1b94819161944147c22e4aaa27b7c"} Jul 14 14:21:36.651808 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:36.651775 2564 generic.go:356] "Generic (PLEG): container finished" podID="65822b17-1b87-4297-b62b-72a98e337166" containerID="002739b70f8b802ec352a6944df9c15c85f1b94819161944147c22e4aaa27b7c" exitCode=0 Jul 14 14:21:36.652267 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:36.651850 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" event={"ID":"65822b17-1b87-4297-b62b-72a98e337166","Type":"ContainerDied","Data":"002739b70f8b802ec352a6944df9c15c85f1b94819161944147c22e4aaa27b7c"} Jul 14 14:21:36.652267 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:36.651923 2564 scope.go:117] "RemoveContainer" containerID="54578f86e053fbe073c0bd18812d6dce27c75e7825d342d46b9cd75a1ebda534" Jul 14 14:21:36.708392 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:36.708349 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r"] Jul 14 14:21:37.788672 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:37.788647 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" Jul 14 14:21:37.812234 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:37.812209 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65822b17-1b87-4297-b62b-72a98e337166-maas-api-service-ca\") pod \"65822b17-1b87-4297-b62b-72a98e337166\" (UID: \"65822b17-1b87-4297-b62b-72a98e337166\") " Jul 14 14:21:37.812585 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:37.812564 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65822b17-1b87-4297-b62b-72a98e337166-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65822b17-1b87-4297-b62b-72a98e337166" (UID: "65822b17-1b87-4297-b62b-72a98e337166"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:21:37.913602 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:37.913512 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65822b17-1b87-4297-b62b-72a98e337166-maas-api-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:21:38.666635 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:38.666604 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" Jul 14 14:21:38.666806 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:38.666642 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r" event={"ID":"65822b17-1b87-4297-b62b-72a98e337166","Type":"ContainerDied","Data":"ce4f7ab880192feb98c9096a3dad955e0395317b9e0168f4b1a80875deb04dc0"} Jul 14 14:21:38.666806 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:38.666690 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce4f7ab880192feb98c9096a3dad955e0395317b9e0168f4b1a80875deb04dc0" Jul 14 14:21:38.698208 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:38.698176 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r"] Jul 14 14:21:38.701247 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:38.701223 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-wvc5r"] Jul 14 14:21:39.370192 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:21:39.370158 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65822b17-1b87-4297-b62b-72a98e337166" path="/var/lib/kubelet/pods/65822b17-1b87-4297-b62b-72a98e337166/volumes" Jul 14 14:22:10.360496 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360461 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg"] Jul 14 14:22:10.360939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360806 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:10.360939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360818 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:10.360939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360841 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:10.360939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360846 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:10.360939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360853 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" Jul 14 14:22:10.360939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360858 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20952eb-f2a3-48fb-95cd-7025cac1a9fc" containerName="istio-proxy" Jul 14 14:22:10.360939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360907 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:10.360939 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.360920 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:10.363387 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.363371 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" Jul 14 14:22:10.366327 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.366303 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nk9bs\"" Jul 14 14:22:10.366327 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.366317 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:22:10.372402 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.372377 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg"] Jul 14 14:22:10.511658 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.511604 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3cdffb-e826-4be9-8f1f-0cac719050ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg\" (UID: \"8f3cdffb-e826-4be9-8f1f-0cac719050ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" Jul 14 14:22:10.612942 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.612834 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3cdffb-e826-4be9-8f1f-0cac719050ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg\" (UID: \"8f3cdffb-e826-4be9-8f1f-0cac719050ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" Jul 14 14:22:10.613510 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.613493 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3cdffb-e826-4be9-8f1f-0cac719050ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg\" (UID: \"8f3cdffb-e826-4be9-8f1f-0cac719050ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" Jul 14 14:22:10.674745 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.674708 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" Jul 14 14:22:10.809387 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.809357 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg"] Jul 14 14:22:10.810229 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:22:10.810199 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f3cdffb_e826_4be9_8f1f_0cac719050ca.slice/crio-97497ecb7e03ca4068b71f288a63566113eb9c26c4de77bfe06a7ade3435dbad WatchSource:0}: Error finding container 97497ecb7e03ca4068b71f288a63566113eb9c26c4de77bfe06a7ade3435dbad: Status 404 returned error can't find the container with id 97497ecb7e03ca4068b71f288a63566113eb9c26c4de77bfe06a7ade3435dbad Jul 14 14:22:10.834771 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:10.834746 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" event={"ID":"8f3cdffb-e826-4be9-8f1f-0cac719050ca","Type":"ContainerStarted","Data":"97497ecb7e03ca4068b71f288a63566113eb9c26c4de77bfe06a7ade3435dbad"} Jul 14 14:22:11.841759 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:11.841724 2564 generic.go:356] "Generic (PLEG): container finished" podID="8f3cdffb-e826-4be9-8f1f-0cac719050ca" containerID="2491c50ab3d0a1bf89543f0731e5439ca8ae1befda0999cae6aab9e2959c42a5" exitCode=0 Jul 14 14:22:11.842151 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:11.841812 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" event={"ID":"8f3cdffb-e826-4be9-8f1f-0cac719050ca","Type":"ContainerDied","Data":"2491c50ab3d0a1bf89543f0731e5439ca8ae1befda0999cae6aab9e2959c42a5"} Jul 14 14:22:12.974378 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:12.974352 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" Jul 14 14:22:13.135788 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:13.135691 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3cdffb-e826-4be9-8f1f-0cac719050ca-maas-api-service-ca\") pod \"8f3cdffb-e826-4be9-8f1f-0cac719050ca\" (UID: \"8f3cdffb-e826-4be9-8f1f-0cac719050ca\") " Jul 14 14:22:13.136097 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:13.136070 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f3cdffb-e826-4be9-8f1f-0cac719050ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f3cdffb-e826-4be9-8f1f-0cac719050ca" (UID: "8f3cdffb-e826-4be9-8f1f-0cac719050ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:22:13.237120 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:13.237085 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f3cdffb-e826-4be9-8f1f-0cac719050ca-maas-api-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:13.854151 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:13.854109 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" event={"ID":"8f3cdffb-e826-4be9-8f1f-0cac719050ca","Type":"ContainerDied","Data":"97497ecb7e03ca4068b71f288a63566113eb9c26c4de77bfe06a7ade3435dbad"} Jul 14 14:22:13.854151 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:13.854140 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg" Jul 14 14:22:13.854396 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:13.854164 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97497ecb7e03ca4068b71f288a63566113eb9c26c4de77bfe06a7ade3435dbad" Jul 14 14:22:20.455054 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:20.455020 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77"] Jul 14 14:22:20.455550 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:20.455301 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" containerID="cri-o://64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a" gracePeriod=30 Jul 14 14:22:25.709578 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.709555 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:22:25.737598 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737503 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-socket\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.737598 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737552 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba62348-0341-4772-b2b8-f0c56f10b69e-istiod-ca-cert\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.737598 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737599 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-podinfo\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.737846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737620 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-envoy\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.737846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737644 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-data\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.737846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737722 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgbsn\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-kube-api-access-bgbsn\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.737846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737761 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-credential-socket\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.737846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737795 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-token\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.737846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737794 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:22:25.737846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.737843 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-certs\") pod \"eba62348-0341-4772-b2b8-f0c56f10b69e\" (UID: \"eba62348-0341-4772-b2b8-f0c56f10b69e\") " Jul 14 14:22:25.738192 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.738002 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba62348-0341-4772-b2b8-f0c56f10b69e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:22:25.738251 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.738139 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.738251 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.738216 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:22:25.738251 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.738234 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eba62348-0341-4772-b2b8-f0c56f10b69e-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.738416 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.738394 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:22:25.738652 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.738627 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-data" (OuterVolumeSpecName: "istio-data") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:22:25.740183 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.740158 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:22:25.740183 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.740209 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:22:25.740506 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.740484 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-kube-api-access-bgbsn" (OuterVolumeSpecName: "kube-api-access-bgbsn") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "kube-api-access-bgbsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:22:25.740604 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.740536 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-token" (OuterVolumeSpecName: "istio-token") pod "eba62348-0341-4772-b2b8-f0c56f10b69e" (UID: "eba62348-0341-4772-b2b8-f0c56f10b69e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:22:25.839358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.839323 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.839358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.839350 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.839358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.839360 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.839358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.839369 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.839632 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.839377 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgbsn\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-kube-api-access-bgbsn\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.839632 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.839387 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eba62348-0341-4772-b2b8-f0c56f10b69e-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.839632 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.839395 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eba62348-0341-4772-b2b8-f0c56f10b69e-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:22:25.917968 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.917932 2564 generic.go:356] "Generic (PLEG): container finished" podID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerID="64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a" exitCode=0 Jul 14 14:22:25.918155 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.917982 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" Jul 14 14:22:25.918155 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.918019 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" event={"ID":"eba62348-0341-4772-b2b8-f0c56f10b69e","Type":"ContainerDied","Data":"64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a"} Jul 14 14:22:25.918155 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.918100 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77" event={"ID":"eba62348-0341-4772-b2b8-f0c56f10b69e","Type":"ContainerDied","Data":"814fea111f7a9425abfdf67e22fb7a61b124a58bf74d48a04c5a8395ab5962b3"} Jul 14 14:22:25.918155 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.918122 2564 scope.go:117] "RemoveContainer" containerID="64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a" Jul 14 14:22:25.927656 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.927635 2564 scope.go:117] "RemoveContainer" containerID="17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f" Jul 14 14:22:25.935854 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.935838 2564 scope.go:117] "RemoveContainer" containerID="64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a" Jul 14 14:22:25.936105 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:22:25.936084 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a\": container with ID starting with 64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a not found: ID does not exist" containerID="64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a" Jul 14 14:22:25.936166 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.936114 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a"} err="failed to get container status \"64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a\": rpc error: code = NotFound desc = could not find container \"64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a\": container with ID starting with 64c9e7e3d915d367972179cae9aed668b6759dc873c8090a38edde0e99a27e3a not found: ID does not exist" Jul 14 14:22:25.936166 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.936132 2564 scope.go:117] "RemoveContainer" containerID="17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f" Jul 14 14:22:25.936390 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:22:25.936367 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f\": container with ID starting with 17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f not found: ID does not exist" containerID="17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f" Jul 14 14:22:25.936477 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.936392 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f"} err="failed to get container status \"17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f\": rpc error: code = NotFound desc = could not find container \"17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f\": container with ID starting with 17e4045290b21c90a25b1f1abc4385e0f5567486d3d43fc1584cded34af70a0f not found: ID does not exist" Jul 14 14:22:25.957672 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.957593 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77"] Jul 14 14:22:25.960241 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:25.960214 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-26a3f6d9-gw-openshift-default-d8695746c-tvr77"] Jul 14 14:22:27.370407 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.370376 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" path="/var/lib/kubelet/pods/eba62348-0341-4772-b2b8-f0c56f10b69e/volumes" Jul 14 14:22:27.697492 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697462 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl"] Jul 14 14:22:27.697855 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697840 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:27.697900 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697857 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:27.697900 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697871 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" Jul 14 14:22:27.697900 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697876 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" Jul 14 14:22:27.697900 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697890 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f3cdffb-e826-4be9-8f1f-0cac719050ca" containerName="revoke-keys" Jul 14 14:22:27.697900 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697896 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f3cdffb-e826-4be9-8f1f-0cac719050ca" containerName="revoke-keys" Jul 14 14:22:27.698051 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697951 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="65822b17-1b87-4297-b62b-72a98e337166" containerName="revoke-keys" Jul 14 14:22:27.698051 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697962 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f3cdffb-e826-4be9-8f1f-0cac719050ca" containerName="revoke-keys" Jul 14 14:22:27.698051 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697969 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" Jul 14 14:22:27.698051 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.697975 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" Jul 14 14:22:27.700899 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.700882 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.704079 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.704058 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6f52b160-openshift-default-dockercfg-bmn75\"" Jul 14 14:22:27.717598 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.717575 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl"] Jul 14 14:22:27.756400 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/306b5607-42c8-474c-8829-83d1ac207607-istiod-ca-cert\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.756610 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756405 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-socket\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.756610 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/306b5607-42c8-474c-8829-83d1ac207607-istio-podinfo\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.756610 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756453 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd9hh\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-kube-api-access-fd9hh\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.756610 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-envoy\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.756610 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756592 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-data\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.756825 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756647 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-certs\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.756825 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756674 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-istio-token\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.756825 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.756694 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-credential-socket\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857169 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/306b5607-42c8-474c-8829-83d1ac207607-istiod-ca-cert\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857169 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857166 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-socket\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857413 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/306b5607-42c8-474c-8829-83d1ac207607-istio-podinfo\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857413 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857320 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd9hh\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-kube-api-access-fd9hh\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857413 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-envoy\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857542 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-data\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857542 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857482 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-certs\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857542 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857507 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-socket\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857542 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857507 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-istio-token\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-credential-socket\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857870 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-data\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857945 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857891 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-certs\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.857945 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857918 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-credential-socket\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.858016 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.857947 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/306b5607-42c8-474c-8829-83d1ac207607-istiod-ca-cert\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.859621 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.859595 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-envoy\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.859832 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.859812 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/306b5607-42c8-474c-8829-83d1ac207607-istio-podinfo\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.867936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.867913 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-istio-token\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:27.868208 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:27.868188 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd9hh\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-kube-api-access-fd9hh\") pod \"e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:28.010947 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:28.010852 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:28.148339 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:28.148309 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl"] Jul 14 14:22:28.149754 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:22:28.149733 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306b5607_42c8_474c_8829_83d1ac207607.slice/crio-01b3b99227895f78d90ca059e9deb38416fc346c548d3d37c0c0abd04f8d1750 WatchSource:0}: Error finding container 01b3b99227895f78d90ca059e9deb38416fc346c548d3d37c0c0abd04f8d1750: Status 404 returned error can't find the container with id 01b3b99227895f78d90ca059e9deb38416fc346c548d3d37c0c0abd04f8d1750 Jul 14 14:22:28.151738 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:28.151706 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:22:28.151824 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:28.151771 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:22:28.151824 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:28.151815 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:22:28.938706 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:28.938672 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" event={"ID":"306b5607-42c8-474c-8829-83d1ac207607","Type":"ContainerStarted","Data":"d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf"} Jul 14 14:22:28.939109 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:28.938757 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" event={"ID":"306b5607-42c8-474c-8829-83d1ac207607","Type":"ContainerStarted","Data":"01b3b99227895f78d90ca059e9deb38416fc346c548d3d37c0c0abd04f8d1750"} Jul 14 14:22:28.967041 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:28.966989 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" podStartSLOduration=1.9669733150000002 podStartE2EDuration="1.966973315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:22:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:22:28.965559695 +0000 UTC m=+1688.192242630" watchObservedRunningTime="2026-07-14 14:22:28.966973315 +0000 UTC m=+1688.193656286" Jul 14 14:22:29.012014 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:29.011977 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:29.017456 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:29.017427 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:29.949240 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:29.946519 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:29.950220 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:29.950202 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:41.006820 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:41.006786 2564 generic.go:356] "Generic (PLEG): container finished" podID="306b5607-42c8-474c-8829-83d1ac207607" containerID="d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf" exitCode=0 Jul 14 14:22:41.007243 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:41.006858 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" event={"ID":"306b5607-42c8-474c-8829-83d1ac207607","Type":"ContainerDied","Data":"d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf"} Jul 14 14:22:41.007243 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:41.007228 2564 scope.go:117] "RemoveContainer" containerID="d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf" Jul 14 14:22:41.008217 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:41.008187 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:22:41.008321 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:41.008251 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:22:41.008321 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:41.008301 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:22:42.013952 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:42.013920 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" event={"ID":"306b5607-42c8-474c-8829-83d1ac207607","Type":"ContainerStarted","Data":"9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657"} Jul 14 14:22:43.011074 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:43.011033 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:43.012239 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:43.012211 2564 patch_prober.go:28] interesting pod/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 14 14:22:43.012354 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:43.012298 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 14 14:22:44.011789 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:44.011751 2564 patch_prober.go:28] interesting pod/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 14 14:22:44.012175 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:44.011818 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 14 14:22:44.947036 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:44.946993 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:45.011434 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:45.011399 2564 patch_prober.go:28] interesting pod/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 14 14:22:45.011608 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:45.011464 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 14 14:22:46.015538 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:46.015509 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:22:46.016563 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:22:46.016544 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:23:00.495156 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.495123 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl"] Jul 14 14:23:00.495592 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.495360 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" containerID="cri-o://9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657" gracePeriod=30 Jul 14 14:23:00.865934 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.865905 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk"] Jul 14 14:23:00.866341 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.866325 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" Jul 14 14:23:00.866341 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.866343 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba62348-0341-4772-b2b8-f0c56f10b69e" containerName="istio-proxy" Jul 14 14:23:00.869342 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.869319 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.869795 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.869770 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-token\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.869923 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.869815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-certs\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.869923 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.869838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-data\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.869923 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.869892 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69n8c\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-kube-api-access-69n8c\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.869923 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.869920 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istiod-ca-cert\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.870067 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.869944 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-envoy\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.870102 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.870071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-credential-socket\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.870135 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.870110 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-socket\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.870189 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.870176 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-podinfo\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.871830 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.871814 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9cad56cb-openshift-default-dockercfg-88vtd\"" Jul 14 14:23:00.890687 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.890656 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk"] Jul 14 14:23:00.971973 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.971937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-podinfo\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.971973 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.971988 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-token\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972010 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-certs\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972025 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-data\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972058 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69n8c\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-kube-api-access-69n8c\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istiod-ca-cert\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972128 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-envoy\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-credential-socket\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972302 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-socket\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972618 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-credential-socket\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972680 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-certs\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.972783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972748 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-data\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.973018 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.972997 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-socket\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.973087 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.973051 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istiod-ca-cert\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.974736 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.974713 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-envoy\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.974831 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.974721 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-podinfo\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.981927 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.981904 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-token\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:00.982042 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:00.981940 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69n8c\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-kube-api-access-69n8c\") pod \"e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:01.018881 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:01.018844 2564 patch_prober.go:28] interesting pod/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 14:23:01.019049 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:01.018912 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 14:23:01.179831 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:01.179743 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:01.323332 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:01.323296 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk"] Jul 14 14:23:01.325529 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:23:01.325498 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1538f51d_8ae1_44e7_8251_6f99c6a637f3.slice/crio-ad653416d0d80dc1880020efadf10f12a4f0ac4231f3ad1337b1a2603687e0d7 WatchSource:0}: Error finding container ad653416d0d80dc1880020efadf10f12a4f0ac4231f3ad1337b1a2603687e0d7: Status 404 returned error can't find the container with id ad653416d0d80dc1880020efadf10f12a4f0ac4231f3ad1337b1a2603687e0d7 Jul 14 14:23:01.327531 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:01.327502 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:23:01.327597 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:01.327568 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:23:01.327634 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:01.327595 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:23:02.134296 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:02.134249 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" event={"ID":"1538f51d-8ae1-44e7-8251-6f99c6a637f3","Type":"ContainerStarted","Data":"6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80"} Jul 14 14:23:02.134696 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:02.134330 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" event={"ID":"1538f51d-8ae1-44e7-8251-6f99c6a637f3","Type":"ContainerStarted","Data":"ad653416d0d80dc1880020efadf10f12a4f0ac4231f3ad1337b1a2603687e0d7"} Jul 14 14:23:02.162887 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:02.162838 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" podStartSLOduration=2.162821412 podStartE2EDuration="2.162821412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:23:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:23:02.160021055 +0000 UTC m=+1721.386703991" watchObservedRunningTime="2026-07-14 14:23:02.162821412 +0000 UTC m=+1721.389504329" Jul 14 14:23:02.180438 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:02.180407 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:02.185351 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:02.185327 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:03.140350 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:03.140319 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:03.141197 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:03.141178 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:05.745085 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.745064 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:23:05.814234 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814203 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-certs\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814427 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814250 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-credential-socket\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814427 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814292 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/306b5607-42c8-474c-8829-83d1ac207607-istiod-ca-cert\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814427 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814336 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-istio-token\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814427 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814355 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd9hh\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-kube-api-access-fd9hh\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814427 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814374 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-envoy\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814427 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814402 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-data\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814748 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814430 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/306b5607-42c8-474c-8829-83d1ac207607-istio-podinfo\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814748 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814467 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-socket\") pod \"306b5607-42c8-474c-8829-83d1ac207607\" (UID: \"306b5607-42c8-474c-8829-83d1ac207607\") " Jul 14 14:23:05.814748 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814527 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:23:05.814748 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814539 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:23:05.814748 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814732 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306b5607-42c8-474c-8829-83d1ac207607-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:23:05.815020 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814781 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:05.815020 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814804 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:05.815020 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.814989 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:23:05.815465 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.815442 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-data" (OuterVolumeSpecName: "istio-data") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:23:05.816947 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.816909 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/306b5607-42c8-474c-8829-83d1ac207607-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:23:05.817165 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.817144 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:23:05.817242 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.817187 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-istio-token" (OuterVolumeSpecName: "istio-token") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:23:05.817510 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.817485 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-kube-api-access-fd9hh" (OuterVolumeSpecName: "kube-api-access-fd9hh") pod "306b5607-42c8-474c-8829-83d1ac207607" (UID: "306b5607-42c8-474c-8829-83d1ac207607"). InnerVolumeSpecName "kube-api-access-fd9hh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:23:05.915784 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.915747 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:05.915784 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.915778 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/306b5607-42c8-474c-8829-83d1ac207607-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:05.915784 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.915788 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:05.916023 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.915796 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd9hh\" (UniqueName: \"kubernetes.io/projected/306b5607-42c8-474c-8829-83d1ac207607-kube-api-access-fd9hh\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:05.916023 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.915807 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:05.916023 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.915815 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/306b5607-42c8-474c-8829-83d1ac207607-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:05.916023 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:05.915822 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/306b5607-42c8-474c-8829-83d1ac207607-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:06.157783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.157686 2564 generic.go:356] "Generic (PLEG): container finished" podID="306b5607-42c8-474c-8829-83d1ac207607" containerID="9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657" exitCode=0 Jul 14 14:23:06.157783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.157744 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" Jul 14 14:23:06.158005 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.157778 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" event={"ID":"306b5607-42c8-474c-8829-83d1ac207607","Type":"ContainerDied","Data":"9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657"} Jul 14 14:23:06.158005 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.157864 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl" event={"ID":"306b5607-42c8-474c-8829-83d1ac207607","Type":"ContainerDied","Data":"01b3b99227895f78d90ca059e9deb38416fc346c548d3d37c0c0abd04f8d1750"} Jul 14 14:23:06.158005 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.157888 2564 scope.go:117] "RemoveContainer" containerID="9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657" Jul 14 14:23:06.167886 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.167868 2564 scope.go:117] "RemoveContainer" containerID="d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf" Jul 14 14:23:06.175799 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.175779 2564 scope.go:117] "RemoveContainer" containerID="9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657" Jul 14 14:23:06.176034 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:23:06.176013 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657\": container with ID starting with 9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657 not found: ID does not exist" containerID="9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657" Jul 14 14:23:06.176034 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.176039 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657"} err="failed to get container status \"9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657\": rpc error: code = NotFound desc = could not find container \"9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657\": container with ID starting with 9a38fbe96131ed5e1af256d41f9bf5b36101c1655104d9a8ba00f89b6a482657 not found: ID does not exist" Jul 14 14:23:06.176194 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.176057 2564 scope.go:117] "RemoveContainer" containerID="d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf" Jul 14 14:23:06.176312 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:23:06.176275 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf\": container with ID starting with d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf not found: ID does not exist" containerID="d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf" Jul 14 14:23:06.176362 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.176321 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf"} err="failed to get container status \"d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf\": rpc error: code = NotFound desc = could not find container \"d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf\": container with ID starting with d159820bf2b11202ac08069503e65eb4e2ea1828ff12f904193c6b4287db63bf not found: ID does not exist" Jul 14 14:23:06.198633 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.198606 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl"] Jul 14 14:23:06.202573 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.202544 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6f52b160-openshift-default-8586886b59-mm8wl"] Jul 14 14:23:06.517037 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.516951 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr"] Jul 14 14:23:06.517413 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.517398 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" Jul 14 14:23:06.517475 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.517415 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" Jul 14 14:23:06.517475 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.517436 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" Jul 14 14:23:06.517475 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.517441 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" Jul 14 14:23:06.517569 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.517501 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" Jul 14 14:23:06.517569 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.517509 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="306b5607-42c8-474c-8829-83d1ac207607" containerName="istio-proxy" Jul 14 14:23:06.522411 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.522395 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" Jul 14 14:23:06.528656 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.528325 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nk9bs\"" Jul 14 14:23:06.528656 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.528325 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:23:06.544512 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.544484 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr"] Jul 14 14:23:06.621912 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.621875 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8a9354-b67f-4a56-8f1f-d7291bf2ee59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr\" (UID: \"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" Jul 14 14:23:06.722537 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.722500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8a9354-b67f-4a56-8f1f-d7291bf2ee59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr\" (UID: \"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" Jul 14 14:23:06.723224 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.723205 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8a9354-b67f-4a56-8f1f-d7291bf2ee59-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr\" (UID: \"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" Jul 14 14:23:06.831869 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.831833 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" Jul 14 14:23:06.969589 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:06.969557 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr"] Jul 14 14:23:06.970999 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:23:06.970973 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8a9354_b67f_4a56_8f1f_d7291bf2ee59.slice/crio-99da831e1b52d5107c5c07dcd6c97315847d7033f8da58f8be6ff868e14392ba WatchSource:0}: Error finding container 99da831e1b52d5107c5c07dcd6c97315847d7033f8da58f8be6ff868e14392ba: Status 404 returned error can't find the container with id 99da831e1b52d5107c5c07dcd6c97315847d7033f8da58f8be6ff868e14392ba Jul 14 14:23:07.163860 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:07.163831 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" event={"ID":"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59","Type":"ContainerStarted","Data":"99da831e1b52d5107c5c07dcd6c97315847d7033f8da58f8be6ff868e14392ba"} Jul 14 14:23:07.370238 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:07.370203 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="306b5607-42c8-474c-8829-83d1ac207607" path="/var/lib/kubelet/pods/306b5607-42c8-474c-8829-83d1ac207607/volumes" Jul 14 14:23:08.171877 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:08.171839 2564 generic.go:356] "Generic (PLEG): container finished" podID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerID="520e6ab44684016a99b16855bfa220800cd543345d007dc5fb3f29eef6b3c8a6" exitCode=7 Jul 14 14:23:08.172269 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:08.171918 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" event={"ID":"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59","Type":"ContainerDied","Data":"520e6ab44684016a99b16855bfa220800cd543345d007dc5fb3f29eef6b3c8a6"} Jul 14 14:23:08.172269 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:08.172095 2564 scope.go:117] "RemoveContainer" containerID="520e6ab44684016a99b16855bfa220800cd543345d007dc5fb3f29eef6b3c8a6" Jul 14 14:23:09.179630 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:09.179599 2564 generic.go:356] "Generic (PLEG): container finished" podID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerID="eea2bd367e54ee6bd4cd0f7b2162c7a15a19e9077fd08633ebaef0dc06a331db" exitCode=7 Jul 14 14:23:09.180038 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:09.179692 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" event={"ID":"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59","Type":"ContainerDied","Data":"eea2bd367e54ee6bd4cd0f7b2162c7a15a19e9077fd08633ebaef0dc06a331db"} Jul 14 14:23:09.180038 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:09.179754 2564 scope.go:117] "RemoveContainer" containerID="520e6ab44684016a99b16855bfa220800cd543345d007dc5fb3f29eef6b3c8a6" Jul 14 14:23:09.180038 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:09.179887 2564 scope.go:117] "RemoveContainer" containerID="eea2bd367e54ee6bd4cd0f7b2162c7a15a19e9077fd08633ebaef0dc06a331db" Jul 14 14:23:09.180160 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:23:09.180138 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr_odh-ai-gateway-infra(cc8a9354-b67f-4a56-8f1f-d7291bf2ee59)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" Jul 14 14:23:10.186751 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:10.186726 2564 scope.go:117] "RemoveContainer" containerID="eea2bd367e54ee6bd4cd0f7b2162c7a15a19e9077fd08633ebaef0dc06a331db" Jul 14 14:23:10.187104 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:23:10.186931 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr_odh-ai-gateway-infra(cc8a9354-b67f-4a56-8f1f-d7291bf2ee59)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" Jul 14 14:23:17.232565 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:17.232486 2564 generic.go:356] "Generic (PLEG): container finished" podID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerID="6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80" exitCode=0 Jul 14 14:23:17.232565 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:17.232535 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" event={"ID":"1538f51d-8ae1-44e7-8251-6f99c6a637f3","Type":"ContainerDied","Data":"6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80"} Jul 14 14:23:17.233018 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:17.232902 2564 scope.go:117] "RemoveContainer" containerID="6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80" Jul 14 14:23:17.233911 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:17.233881 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:23:17.233967 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:17.233947 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:23:17.234007 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:17.233979 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:23:18.141207 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:18.141162 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:18.246266 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:18.246223 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" event={"ID":"1538f51d-8ae1-44e7-8251-6f99c6a637f3","Type":"ContainerStarted","Data":"8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4"} Jul 14 14:23:19.180533 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:19.180500 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:19.181627 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:19.181604 2564 patch_prober.go:28] interesting pod/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 14 14:23:19.181691 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:19.181654 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 14 14:23:20.180229 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:20.180194 2564 patch_prober.go:28] interesting pod/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 14 14:23:20.180620 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:20.180251 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 14 14:23:21.180258 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:21.180220 2564 patch_prober.go:28] interesting pod/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 14 14:23:21.180675 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:21.180274 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 14 14:23:22.184412 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:22.184383 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:22.184810 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:22.184656 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:22.185485 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:22.185463 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:23:22.364423 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:22.364394 2564 scope.go:117] "RemoveContainer" containerID="eea2bd367e54ee6bd4cd0f7b2162c7a15a19e9077fd08633ebaef0dc06a331db" Jul 14 14:23:23.277662 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:23.277625 2564 generic.go:356] "Generic (PLEG): container finished" podID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerID="84774d950ee9c57cfde36984f3acfb68b30689478a64e412dc77899273360817" exitCode=0 Jul 14 14:23:23.278102 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:23.277674 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" event={"ID":"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59","Type":"ContainerDied","Data":"84774d950ee9c57cfde36984f3acfb68b30689478a64e412dc77899273360817"} Jul 14 14:23:23.278102 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:23.277766 2564 scope.go:117] "RemoveContainer" containerID="eea2bd367e54ee6bd4cd0f7b2162c7a15a19e9077fd08633ebaef0dc06a331db" Jul 14 14:23:24.313205 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:24.313164 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr"] Jul 14 14:23:24.420113 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:24.420087 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" Jul 14 14:23:24.493718 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:24.493686 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8a9354-b67f-4a56-8f1f-d7291bf2ee59-maas-api-service-ca\") pod \"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59\" (UID: \"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59\") " Jul 14 14:23:24.494038 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:24.494015 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc8a9354-b67f-4a56-8f1f-d7291bf2ee59-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" (UID: "cc8a9354-b67f-4a56-8f1f-d7291bf2ee59"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:23:24.594697 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:24.594667 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc8a9354-b67f-4a56-8f1f-d7291bf2ee59-maas-api-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:23:25.291111 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:25.291081 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" Jul 14 14:23:25.291301 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:25.291115 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr" event={"ID":"cc8a9354-b67f-4a56-8f1f-d7291bf2ee59","Type":"ContainerDied","Data":"99da831e1b52d5107c5c07dcd6c97315847d7033f8da58f8be6ff868e14392ba"} Jul 14 14:23:25.291301 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:25.291162 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99da831e1b52d5107c5c07dcd6c97315847d7033f8da58f8be6ff868e14392ba" Jul 14 14:23:25.338594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:25.338563 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr"] Jul 14 14:23:25.344004 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:25.343983 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9cad56cb-x47nr"] Jul 14 14:23:25.370715 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:23:25.370689 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" path="/var/lib/kubelet/pods/cc8a9354-b67f-4a56-8f1f-d7291bf2ee59/volumes" Jul 14 14:24:21.401540 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:21.401508 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:24:21.402976 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:21.402952 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:24:21.823763 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:21.823728 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk"] Jul 14 14:24:21.823983 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:21.823958 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" containerID="cri-o://8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4" gracePeriod=30 Jul 14 14:24:22.187661 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:22.187574 2564 patch_prober.go:28] interesting pod/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 14:24:22.187661 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:22.187644 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 14:24:27.078727 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.078692 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:24:27.170685 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170651 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-certs\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.170851 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170697 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69n8c\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-kube-api-access-69n8c\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.170851 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170748 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-data\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.170851 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170774 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-token\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.170851 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170809 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-credential-socket\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.170851 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170833 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-podinfo\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.171110 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170872 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istiod-ca-cert\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.171110 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170916 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-envoy\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.171110 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170951 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:24:27.171110 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.170976 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-socket\") pod \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\" (UID: \"1538f51d-8ae1-44e7-8251-6f99c6a637f3\") " Jul 14 14:24:27.171342 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.171234 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:24:27.171342 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.171336 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.171462 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.171341 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:24:27.171462 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.171356 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.171571 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.171465 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:24:27.171744 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.171692 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:24:27.173051 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.173027 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-kube-api-access-69n8c" (OuterVolumeSpecName: "kube-api-access-69n8c") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "kube-api-access-69n8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:24:27.173164 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.173110 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:24:27.173535 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.173512 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:24:27.173609 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.173543 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1538f51d-8ae1-44e7-8251-6f99c6a637f3" (UID: "1538f51d-8ae1-44e7-8251-6f99c6a637f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:24:27.271977 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.271940 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.271977 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.271976 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69n8c\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-kube-api-access-69n8c\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.271977 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.271986 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.271977 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.271994 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.272260 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.272002 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.272260 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.272010 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.272260 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.272018 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1538f51d-8ae1-44e7-8251-6f99c6a637f3-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:24:27.625532 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.625498 2564 generic.go:356] "Generic (PLEG): container finished" podID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerID="8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4" exitCode=0 Jul 14 14:24:27.625704 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.625546 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" Jul 14 14:24:27.625704 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.625577 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" event={"ID":"1538f51d-8ae1-44e7-8251-6f99c6a637f3","Type":"ContainerDied","Data":"8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4"} Jul 14 14:24:27.625704 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.625635 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk" event={"ID":"1538f51d-8ae1-44e7-8251-6f99c6a637f3","Type":"ContainerDied","Data":"ad653416d0d80dc1880020efadf10f12a4f0ac4231f3ad1337b1a2603687e0d7"} Jul 14 14:24:27.625704 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.625650 2564 scope.go:117] "RemoveContainer" containerID="8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4" Jul 14 14:24:27.635054 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.635028 2564 scope.go:117] "RemoveContainer" containerID="6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80" Jul 14 14:24:27.643657 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.643635 2564 scope.go:117] "RemoveContainer" containerID="8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4" Jul 14 14:24:27.643964 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:24:27.643943 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4\": container with ID starting with 8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4 not found: ID does not exist" containerID="8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4" Jul 14 14:24:27.644035 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.643976 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4"} err="failed to get container status \"8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4\": rpc error: code = NotFound desc = could not find container \"8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4\": container with ID starting with 8cc23f2ddcb8304f56f0ac743ee7e5b8a501f7eccc7ad8449b41e1a3be2156f4 not found: ID does not exist" Jul 14 14:24:27.644035 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.643997 2564 scope.go:117] "RemoveContainer" containerID="6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80" Jul 14 14:24:27.644433 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:24:27.644372 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80\": container with ID starting with 6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80 not found: ID does not exist" containerID="6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80" Jul 14 14:24:27.644433 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.644405 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80"} err="failed to get container status \"6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80\": rpc error: code = NotFound desc = could not find container \"6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80\": container with ID starting with 6a028013aa1ed331c5fb718af19f84a1af46d8cd6b6cae5e86d4d2a395245c80 not found: ID does not exist" Jul 14 14:24:27.647391 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.647371 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk"] Jul 14 14:24:27.651195 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:27.651169 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9cad56cb-openshift-default-578ddbffb7-q75qk"] Jul 14 14:24:29.370033 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:24:29.369998 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" path="/var/lib/kubelet/pods/1538f51d-8ae1-44e7-8251-6f99c6a637f3/volumes" Jul 14 14:26:02.982145 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982063 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm"] Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982425 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982437 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982447 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982453 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982459 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982465 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982476 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982484 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982510 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982518 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982580 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982589 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982595 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982603 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc8a9354-b67f-4a56-8f1f-d7291bf2ee59" containerName="revoke-keys" Jul 14 14:26:02.982631 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.982610 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1538f51d-8ae1-44e7-8251-6f99c6a637f3" containerName="istio-proxy" Jul 14 14:26:02.985605 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.985587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:02.989795 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:02.989774 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-32d459c1-openshift-default-dockercfg-cqh8m\"" Jul 14 14:26:03.015092 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.015066 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm"] Jul 14 14:26:03.131564 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-envoy\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.131738 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131580 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11956029-4bd2-443c-81ac-455143fb7eee-istio-podinfo\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.131738 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131604 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-istio-token\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.131738 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11956029-4bd2-443c-81ac-455143fb7eee-istiod-ca-cert\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.131738 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131686 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-certs\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.131738 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131736 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-socket\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.132000 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131829 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njn44\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-kube-api-access-njn44\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.132000 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131876 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-credential-socket\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.132000 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.131905 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-data\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233187 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11956029-4bd2-443c-81ac-455143fb7eee-istio-podinfo\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233187 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233141 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-istio-token\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233187 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233166 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11956029-4bd2-443c-81ac-455143fb7eee-istiod-ca-cert\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233187 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233189 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-certs\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233538 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-socket\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233538 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njn44\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-kube-api-access-njn44\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233538 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-credential-socket\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233538 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-data\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233538 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-envoy\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233797 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-certs\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233797 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-socket\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233900 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233845 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-data\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233952 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-credential-socket\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.233991 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.233976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11956029-4bd2-443c-81ac-455143fb7eee-istiod-ca-cert\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.235620 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.235599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-envoy\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.235777 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.235758 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11956029-4bd2-443c-81ac-455143fb7eee-istio-podinfo\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.252560 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.252535 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-istio-token\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.252785 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.252764 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njn44\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-kube-api-access-njn44\") pod \"e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.296644 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.296615 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:03.461719 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.461685 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm"] Jul 14 14:26:03.463371 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:26:03.463343 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11956029_4bd2_443c_81ac_455143fb7eee.slice/crio-16ab4dac4998a21fa43c9a9bc20655150f1b4bac6c4cbba7fa124fe4f0dfeb03 WatchSource:0}: Error finding container 16ab4dac4998a21fa43c9a9bc20655150f1b4bac6c4cbba7fa124fe4f0dfeb03: Status 404 returned error can't find the container with id 16ab4dac4998a21fa43c9a9bc20655150f1b4bac6c4cbba7fa124fe4f0dfeb03 Jul 14 14:26:03.465552 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.465518 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:26:03.465644 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.465588 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:26:03.465644 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:03.465615 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:26:04.142559 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:04.142524 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" event={"ID":"11956029-4bd2-443c-81ac-455143fb7eee","Type":"ContainerStarted","Data":"8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f"} Jul 14 14:26:04.142951 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:04.142577 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" event={"ID":"11956029-4bd2-443c-81ac-455143fb7eee","Type":"ContainerStarted","Data":"16ab4dac4998a21fa43c9a9bc20655150f1b4bac6c4cbba7fa124fe4f0dfeb03"} Jul 14 14:26:04.168334 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:04.168271 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" podStartSLOduration=2.168255988 podStartE2EDuration="2.168255988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:26:04.165935147 +0000 UTC m=+1903.392618063" watchObservedRunningTime="2026-07-14 14:26:04.168255988 +0000 UTC m=+1903.394938904" Jul 14 14:26:04.297468 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:04.297434 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:04.302225 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:04.302200 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:05.147871 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:05.147838 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:05.148701 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:05.148683 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:20.270695 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:20.270236 2564 patch_prober.go:28] interesting pod/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": read tcp 10.134.0.2:46246->10.134.0.49:15021: read: connection reset by peer" start-of-body= Jul 14 14:26:20.270695 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:20.270374 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": read tcp 10.134.0.2:46246->10.134.0.49:15021: read: connection reset by peer" Jul 14 14:26:21.235117 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:21.235084 2564 generic.go:356] "Generic (PLEG): container finished" podID="11956029-4bd2-443c-81ac-455143fb7eee" containerID="8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f" exitCode=0 Jul 14 14:26:21.235307 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:21.235166 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" event={"ID":"11956029-4bd2-443c-81ac-455143fb7eee","Type":"ContainerDied","Data":"8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f"} Jul 14 14:26:21.235608 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:21.235592 2564 scope.go:117] "RemoveContainer" containerID="8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f" Jul 14 14:26:21.236543 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:21.236511 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:26:21.236595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:21.236583 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:26:21.236831 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:21.236636 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:26:21.457873 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:21.457840 2564 scope.go:117] "RemoveContainer" containerID="dce1483ba7371a64d17b94aa8f7cc0bbe5e1036e462c79c5f6702da462fb924b" Jul 14 14:26:22.243007 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:22.242973 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" event={"ID":"11956029-4bd2-443c-81ac-455143fb7eee","Type":"ContainerStarted","Data":"80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b"} Jul 14 14:26:22.297100 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:22.297070 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:22.297365 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:22.297339 2564 patch_prober.go:28] interesting pod/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:26:22.297485 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:22.297382 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:26:23.297930 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:23.297892 2564 patch_prober.go:28] interesting pod/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:26:23.298324 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:23.297952 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:26:24.297136 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:24.297097 2564 patch_prober.go:28] interesting pod/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:26:24.297334 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:24.297165 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:26:25.297660 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:25.297627 2564 patch_prober.go:28] interesting pod/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:26:25.298063 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:25.297694 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:26:26.297815 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:26.297774 2564 patch_prober.go:28] interesting pod/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 14 14:26:26.298206 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:26.297845 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 14 14:26:27.300612 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:27.300584 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:27.300976 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:27.300925 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:27.301615 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:27.301594 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:44.821889 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:44.821855 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm"] Jul 14 14:26:44.822413 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:44.822068 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" containerID="cri-o://80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b" gracePeriod=30 Jul 14 14:26:50.076031 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.076002 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:50.147855 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.147787 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-socket\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.147855 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.147826 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njn44\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-kube-api-access-njn44\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.147855 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.147847 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-data\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.148069 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.147885 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-credential-socket\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.148069 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.147933 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-istio-token\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.148069 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.147973 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-certs\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.148069 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148008 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11956029-4bd2-443c-81ac-455143fb7eee-istiod-ca-cert\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.148069 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148037 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11956029-4bd2-443c-81ac-455143fb7eee-istio-podinfo\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.148349 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148080 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-envoy\") pod \"11956029-4bd2-443c-81ac-455143fb7eee\" (UID: \"11956029-4bd2-443c-81ac-455143fb7eee\") " Jul 14 14:26:50.148349 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148080 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:26:50.148349 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148327 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:26:50.148546 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148516 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11956029-4bd2-443c-81ac-455143fb7eee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:26:50.148616 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148560 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.148616 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148581 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.148616 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148596 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:26:50.148799 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.148770 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-data" (OuterVolumeSpecName: "istio-data") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:26:50.150087 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.150066 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:26:50.150179 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.150089 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-kube-api-access-njn44" (OuterVolumeSpecName: "kube-api-access-njn44") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "kube-api-access-njn44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:26:50.150375 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.150360 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-istio-token" (OuterVolumeSpecName: "istio-token") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:26:50.150440 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.150422 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11956029-4bd2-443c-81ac-455143fb7eee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11956029-4bd2-443c-81ac-455143fb7eee" (UID: "11956029-4bd2-443c-81ac-455143fb7eee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:26:50.249251 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.249218 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njn44\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-kube-api-access-njn44\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.249251 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.249245 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.249251 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.249254 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.249463 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.249262 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11956029-4bd2-443c-81ac-455143fb7eee-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.249463 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.249271 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11956029-4bd2-443c-81ac-455143fb7eee-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.249463 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.249290 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11956029-4bd2-443c-81ac-455143fb7eee-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.249463 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.249298 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11956029-4bd2-443c-81ac-455143fb7eee-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:26:50.391076 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.391044 2564 generic.go:356] "Generic (PLEG): container finished" podID="11956029-4bd2-443c-81ac-455143fb7eee" containerID="80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b" exitCode=0 Jul 14 14:26:50.391254 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.391095 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" Jul 14 14:26:50.391254 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.391126 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" event={"ID":"11956029-4bd2-443c-81ac-455143fb7eee","Type":"ContainerDied","Data":"80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b"} Jul 14 14:26:50.391254 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.391172 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm" event={"ID":"11956029-4bd2-443c-81ac-455143fb7eee","Type":"ContainerDied","Data":"16ab4dac4998a21fa43c9a9bc20655150f1b4bac6c4cbba7fa124fe4f0dfeb03"} Jul 14 14:26:50.391254 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.391194 2564 scope.go:117] "RemoveContainer" containerID="80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b" Jul 14 14:26:50.400434 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.400417 2564 scope.go:117] "RemoveContainer" containerID="8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f" Jul 14 14:26:50.408484 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.408465 2564 scope.go:117] "RemoveContainer" containerID="80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b" Jul 14 14:26:50.408731 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:26:50.408712 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b\": container with ID starting with 80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b not found: ID does not exist" containerID="80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b" Jul 14 14:26:50.408785 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.408739 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b"} err="failed to get container status \"80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b\": rpc error: code = NotFound desc = could not find container \"80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b\": container with ID starting with 80f322dc12d151da6c73221590222f185679c26ccb017e3336e94520fdd5033b not found: ID does not exist" Jul 14 14:26:50.408785 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.408757 2564 scope.go:117] "RemoveContainer" containerID="8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f" Jul 14 14:26:50.408997 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:26:50.408980 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f\": container with ID starting with 8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f not found: ID does not exist" containerID="8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f" Jul 14 14:26:50.409054 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.409002 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f"} err="failed to get container status \"8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f\": rpc error: code = NotFound desc = could not find container \"8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f\": container with ID starting with 8ccb1693b17d28a7466b01c40fbb7374fb75209cc5139ddb2900c79ae9e5740f not found: ID does not exist" Jul 14 14:26:50.419359 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.419327 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm"] Jul 14 14:26:50.421331 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:50.421310 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-32d459c1-openshift-default-5454668c9f-w92rm"] Jul 14 14:26:51.370288 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:26:51.370246 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11956029-4bd2-443c-81ac-455143fb7eee" path="/var/lib/kubelet/pods/11956029-4bd2-443c-81ac-455143fb7eee/volumes" Jul 14 14:27:05.403843 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.403808 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv"] Jul 14 14:27:05.404198 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.404178 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" Jul 14 14:27:05.404198 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.404194 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" Jul 14 14:27:05.404274 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.404207 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" Jul 14 14:27:05.404274 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.404214 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" Jul 14 14:27:05.404389 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.404294 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" Jul 14 14:27:05.404389 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.404306 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="11956029-4bd2-443c-81ac-455143fb7eee" containerName="istio-proxy" Jul 14 14:27:05.407328 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.407311 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.410695 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.410672 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5122206c-openshift-default-dockercfg-nv46f\"" Jul 14 14:27:05.423416 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.423390 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv"] Jul 14 14:27:05.592394 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592358 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-token\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.592584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7gtj\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-kube-api-access-v7gtj\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.592584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592442 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-certs\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.592584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592462 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-podinfo\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.592584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592478 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-data\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.592817 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592600 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istiod-ca-cert\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.592817 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592650 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-credential-socket\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.592817 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592693 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-envoy\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.592817 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.592737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-socket\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.693646 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693571 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-credential-socket\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.693646 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693605 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-envoy\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.693646 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693627 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-socket\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.693906 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-token\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.693906 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7gtj\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-kube-api-access-v7gtj\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.693906 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-certs\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.693906 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693855 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-podinfo\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.693906 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693898 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-data\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.694154 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693966 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-credential-socket\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.694154 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.693993 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istiod-ca-cert\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.694154 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.694138 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-socket\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.694303 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.694258 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-data\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.694401 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.694383 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-certs\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.694568 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.694552 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istiod-ca-cert\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.695924 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.695900 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-envoy\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.696404 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.696383 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-podinfo\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.703870 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.703848 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-token\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.703977 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.703886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7gtj\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-kube-api-access-v7gtj\") pod \"e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.717809 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.717790 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:05.858021 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.857992 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv"] Jul 14 14:27:05.860478 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:27:05.860451 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f900d1_d01f_44e5_8005_0ad3f7aedc9c.slice/crio-5ee41cb71e7552a6a1dea2718baccd3ae087ee7476251b1b5c11878e568b94f2 WatchSource:0}: Error finding container 5ee41cb71e7552a6a1dea2718baccd3ae087ee7476251b1b5c11878e568b94f2: Status 404 returned error can't find the container with id 5ee41cb71e7552a6a1dea2718baccd3ae087ee7476251b1b5c11878e568b94f2 Jul 14 14:27:05.863105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.863077 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:27:05.863177 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.863138 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:27:05.863177 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:05.863168 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:27:06.479636 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:06.479598 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" event={"ID":"21f900d1-d01f-44e5-8005-0ad3f7aedc9c","Type":"ContainerStarted","Data":"83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717"} Jul 14 14:27:06.480048 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:06.479649 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" event={"ID":"21f900d1-d01f-44e5-8005-0ad3f7aedc9c","Type":"ContainerStarted","Data":"5ee41cb71e7552a6a1dea2718baccd3ae087ee7476251b1b5c11878e568b94f2"} Jul 14 14:27:06.501039 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:06.500989 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" podStartSLOduration=1.500971149 podStartE2EDuration="1.500971149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:27:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:27:06.500274529 +0000 UTC m=+1965.726957458" watchObservedRunningTime="2026-07-14 14:27:06.500971149 +0000 UTC m=+1965.727654067" Jul 14 14:27:06.718947 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:06.718915 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:06.723590 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:06.723568 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:07.484511 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:07.484483 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:07.485514 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:07.485494 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:22.488952 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:22.488916 2564 patch_prober.go:28] interesting pod/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:52870->10.134.0.50:15021: read: connection reset by peer" start-of-body= Jul 14 14:27:22.489523 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:22.488987 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": read tcp 10.134.0.2:52870->10.134.0.50:15021: read: connection reset by peer" Jul 14 14:27:22.577266 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:22.577237 2564 generic.go:356] "Generic (PLEG): container finished" podID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerID="83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717" exitCode=0 Jul 14 14:27:22.577408 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:22.577315 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" event={"ID":"21f900d1-d01f-44e5-8005-0ad3f7aedc9c","Type":"ContainerDied","Data":"83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717"} Jul 14 14:27:22.577747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:22.577732 2564 scope.go:117] "RemoveContainer" containerID="83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717" Jul 14 14:27:22.578772 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:22.578748 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:27:22.578833 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:22.578812 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:27:22.578871 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:22.578844 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:27:23.584273 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:23.584240 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" event={"ID":"21f900d1-d01f-44e5-8005-0ad3f7aedc9c","Type":"ContainerStarted","Data":"41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19"} Jul 14 14:27:23.718622 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:23.718591 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:23.718786 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:23.718738 2564 patch_prober.go:28] interesting pod/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 14 14:27:23.718835 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:23.718785 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 14 14:27:24.719154 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:24.719121 2564 patch_prober.go:28] interesting pod/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 14 14:27:24.719548 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:24.719182 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 14 14:27:25.718883 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:25.718848 2564 patch_prober.go:28] interesting pod/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 14 14:27:25.719070 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:25.718898 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 14 14:27:26.719022 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:26.718984 2564 patch_prober.go:28] interesting pod/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 14 14:27:26.719425 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:26.719054 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 14 14:27:27.722332 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:27.722297 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:27.722801 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:27.722684 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:27.723448 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:27.723430 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:27:51.812738 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:51.812704 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp"] Jul 14 14:27:51.816471 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:51.816448 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" Jul 14 14:27:51.820484 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:51.820463 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nk9bs\"" Jul 14 14:27:51.821273 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:51.821255 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 14:27:51.831895 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:51.831872 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp"] Jul 14 14:27:51.986897 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:51.986867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f005a1ad-d64b-41af-8fcb-4e339bde6dca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5122206c-dkzmp\" (UID: \"f005a1ad-d64b-41af-8fcb-4e339bde6dca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" Jul 14 14:27:52.088253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:52.088165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f005a1ad-d64b-41af-8fcb-4e339bde6dca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5122206c-dkzmp\" (UID: \"f005a1ad-d64b-41af-8fcb-4e339bde6dca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" Jul 14 14:27:52.088851 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:52.088827 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f005a1ad-d64b-41af-8fcb-4e339bde6dca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5122206c-dkzmp\" (UID: \"f005a1ad-d64b-41af-8fcb-4e339bde6dca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" Jul 14 14:27:52.128487 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:52.128461 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" Jul 14 14:27:52.257485 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:52.257453 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp"] Jul 14 14:27:52.258417 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:27:52.258390 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf005a1ad_d64b_41af_8fcb_4e339bde6dca.slice/crio-9d34c2c3739307f7c36f3cf26215e630ffac868d60d1b2c5fb22b68baf911c2c WatchSource:0}: Error finding container 9d34c2c3739307f7c36f3cf26215e630ffac868d60d1b2c5fb22b68baf911c2c: Status 404 returned error can't find the container with id 9d34c2c3739307f7c36f3cf26215e630ffac868d60d1b2c5fb22b68baf911c2c Jul 14 14:27:52.736257 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:52.736164 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" event={"ID":"f005a1ad-d64b-41af-8fcb-4e339bde6dca","Type":"ContainerStarted","Data":"9d1d98df3a2d5fc21c99d5ea6411aacc77da4dc199b6b7561bc52a4fc871985d"} Jul 14 14:27:52.736257 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:52.736226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" event={"ID":"f005a1ad-d64b-41af-8fcb-4e339bde6dca","Type":"ContainerStarted","Data":"9d34c2c3739307f7c36f3cf26215e630ffac868d60d1b2c5fb22b68baf911c2c"} Jul 14 14:27:52.753943 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:52.753897 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" podStartSLOduration=1.753881499 podStartE2EDuration="1.753881499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:27:52.751796703 +0000 UTC m=+2011.978479642" watchObservedRunningTime="2026-07-14 14:27:52.753881499 +0000 UTC m=+2011.980564445" Jul 14 14:27:53.742677 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:53.742641 2564 generic.go:356] "Generic (PLEG): container finished" podID="f005a1ad-d64b-41af-8fcb-4e339bde6dca" containerID="9d1d98df3a2d5fc21c99d5ea6411aacc77da4dc199b6b7561bc52a4fc871985d" exitCode=0 Jul 14 14:27:53.743044 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:53.742694 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" event={"ID":"f005a1ad-d64b-41af-8fcb-4e339bde6dca","Type":"ContainerDied","Data":"9d1d98df3a2d5fc21c99d5ea6411aacc77da4dc199b6b7561bc52a4fc871985d"} Jul 14 14:27:54.881573 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:54.881551 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" Jul 14 14:27:55.013060 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:55.012975 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f005a1ad-d64b-41af-8fcb-4e339bde6dca-maas-api-service-ca\") pod \"f005a1ad-d64b-41af-8fcb-4e339bde6dca\" (UID: \"f005a1ad-d64b-41af-8fcb-4e339bde6dca\") " Jul 14 14:27:55.013339 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:55.013314 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f005a1ad-d64b-41af-8fcb-4e339bde6dca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f005a1ad-d64b-41af-8fcb-4e339bde6dca" (UID: "f005a1ad-d64b-41af-8fcb-4e339bde6dca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:27:55.114643 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:55.114611 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f005a1ad-d64b-41af-8fcb-4e339bde6dca-maas-api-service-ca\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:27:55.754626 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:55.754538 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" Jul 14 14:27:55.754626 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:55.754578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5122206c-dkzmp" event={"ID":"f005a1ad-d64b-41af-8fcb-4e339bde6dca","Type":"ContainerDied","Data":"9d34c2c3739307f7c36f3cf26215e630ffac868d60d1b2c5fb22b68baf911c2c"} Jul 14 14:27:55.754875 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:27:55.754630 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d34c2c3739307f7c36f3cf26215e630ffac868d60d1b2c5fb22b68baf911c2c" Jul 14 14:28:21.496261 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:28:21.496227 2564 scope.go:117] "RemoveContainer" containerID="002739b70f8b802ec352a6944df9c15c85f1b94819161944147c22e4aaa27b7c" Jul 14 14:28:59.746738 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:28:59.746658 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv"] Jul 14 14:28:59.747303 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:28:59.746863 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" containerID="cri-o://41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19" gracePeriod=30 Jul 14 14:29:04.998508 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:04.998484 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:29:05.022600 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022520 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-podinfo\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.022600 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022552 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-token\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.022600 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022574 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-certs\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.022868 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022639 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-credential-socket\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.022868 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022660 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istiod-ca-cert\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.022868 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022697 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-data\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.022868 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022720 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-envoy\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.022868 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022749 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-socket\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.022868 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.022776 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7gtj\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-kube-api-access-v7gtj\") pod \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\" (UID: \"21f900d1-d01f-44e5-8005-0ad3f7aedc9c\") " Jul 14 14:29:05.023190 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.023013 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:29:05.023250 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.023224 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:29:05.023497 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.023471 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:29:05.023702 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.023673 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:29:05.023808 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.023753 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:29:05.024951 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.024928 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:29:05.025230 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.025209 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:29:05.025315 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.025296 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-kube-api-access-v7gtj" (OuterVolumeSpecName: "kube-api-access-v7gtj") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "kube-api-access-v7gtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:29:05.025427 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.025408 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "21f900d1-d01f-44e5-8005-0ad3f7aedc9c" (UID: "21f900d1-d01f-44e5-8005-0ad3f7aedc9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:29:05.124068 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124033 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.124068 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124062 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.124068 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124071 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.124348 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124079 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.124348 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124091 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.124348 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124100 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.124348 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124107 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.124348 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124115 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.124348 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.124124 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7gtj\" (UniqueName: \"kubernetes.io/projected/21f900d1-d01f-44e5-8005-0ad3f7aedc9c-kube-api-access-v7gtj\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:29:05.134386 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.134349 2564 generic.go:356] "Generic (PLEG): container finished" podID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerID="41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19" exitCode=0 Jul 14 14:29:05.134567 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.134400 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" Jul 14 14:29:05.134567 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.134439 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" event={"ID":"21f900d1-d01f-44e5-8005-0ad3f7aedc9c","Type":"ContainerDied","Data":"41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19"} Jul 14 14:29:05.134567 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.134498 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv" event={"ID":"21f900d1-d01f-44e5-8005-0ad3f7aedc9c","Type":"ContainerDied","Data":"5ee41cb71e7552a6a1dea2718baccd3ae087ee7476251b1b5c11878e568b94f2"} Jul 14 14:29:05.134567 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.134522 2564 scope.go:117] "RemoveContainer" containerID="41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19" Jul 14 14:29:05.144531 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.144510 2564 scope.go:117] "RemoveContainer" containerID="83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717" Jul 14 14:29:05.152568 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.152549 2564 scope.go:117] "RemoveContainer" containerID="41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19" Jul 14 14:29:05.152833 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:29:05.152814 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19\": container with ID starting with 41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19 not found: ID does not exist" containerID="41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19" Jul 14 14:29:05.152902 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.152845 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19"} err="failed to get container status \"41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19\": rpc error: code = NotFound desc = could not find container \"41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19\": container with ID starting with 41579d67c5f6559cd24cc3a989315f8410c3f156893e9838fe9a18d0b335ce19 not found: ID does not exist" Jul 14 14:29:05.152902 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.152869 2564 scope.go:117] "RemoveContainer" containerID="83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717" Jul 14 14:29:05.153137 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:29:05.153103 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717\": container with ID starting with 83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717 not found: ID does not exist" containerID="83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717" Jul 14 14:29:05.153180 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.153142 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717"} err="failed to get container status \"83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717\": rpc error: code = NotFound desc = could not find container \"83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717\": container with ID starting with 83b47fe91395a45021fb1fc4e24f752de49585fd414f4d54b9cb20d158415717 not found: ID does not exist" Jul 14 14:29:05.179087 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.179065 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv"] Jul 14 14:29:05.185226 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.185203 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5122206c-openshift-default-6c4bbf6795-8gkxv"] Jul 14 14:29:05.369696 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:05.369663 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" path="/var/lib/kubelet/pods/21f900d1-d01f-44e5-8005-0ad3f7aedc9c/volumes" Jul 14 14:29:21.440706 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:21.440682 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:29:21.443421 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:29:21.443398 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:30:00.166520 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.166486 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6"] Jul 14 14:30:00.167106 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.166995 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" Jul 14 14:30:00.167106 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.167016 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" Jul 14 14:30:00.167106 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.167029 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f005a1ad-d64b-41af-8fcb-4e339bde6dca" containerName="revoke-keys" Jul 14 14:30:00.167106 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.167038 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f005a1ad-d64b-41af-8fcb-4e339bde6dca" containerName="revoke-keys" Jul 14 14:30:00.167106 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.167067 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" Jul 14 14:30:00.167106 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.167076 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" Jul 14 14:30:00.167458 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.167155 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f005a1ad-d64b-41af-8fcb-4e339bde6dca" containerName="revoke-keys" Jul 14 14:30:00.167458 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.167167 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" Jul 14 14:30:00.167458 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.167178 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="21f900d1-d01f-44e5-8005-0ad3f7aedc9c" containerName="istio-proxy" Jul 14 14:30:00.170338 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.170317 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6" Jul 14 14:30:00.173516 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.173496 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nk9bs\"" Jul 14 14:30:00.182727 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.182390 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6"] Jul 14 14:30:00.184617 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.184593 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6" Jul 14 14:30:00.322413 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:30:00.322375 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9221b79f_5d96_4432_9fdf_7f3febf7ad6c.slice/crio-f709c3696cb8063e8b16123b46e7db2a2a18a5f66d9de8b9c500f8d83766686c WatchSource:0}: Error finding container f709c3696cb8063e8b16123b46e7db2a2a18a5f66d9de8b9c500f8d83766686c: Status 404 returned error can't find the container with id f709c3696cb8063e8b16123b46e7db2a2a18a5f66d9de8b9c500f8d83766686c Jul 14 14:30:00.322776 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.322756 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6"] Jul 14 14:30:00.422454 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:00.422384 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6" event={"ID":"9221b79f-5d96-4432-9fdf-7f3febf7ad6c","Type":"ContainerStarted","Data":"f709c3696cb8063e8b16123b46e7db2a2a18a5f66d9de8b9c500f8d83766686c"} Jul 14 14:30:01.430440 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.430349 2564 generic.go:356] "Generic (PLEG): container finished" podID="9221b79f-5d96-4432-9fdf-7f3febf7ad6c" containerID="fd9f9518af9e3734d03b86fe53b655e87079e057ab60ce05283599e1af299b0e" exitCode=0 Jul 14 14:30:01.430771 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.430434 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6" event={"ID":"9221b79f-5d96-4432-9fdf-7f3febf7ad6c","Type":"ContainerDied","Data":"fd9f9518af9e3734d03b86fe53b655e87079e057ab60ce05283599e1af299b0e"} Jul 14 14:30:01.458462 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.458436 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk"] Jul 14 14:30:01.461775 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.461760 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.464926 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.464903 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-0bd8ad-openshift-default-dockercfg-kp8v5\"" Jul 14 14:30:01.473994 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.473973 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk"] Jul 14 14:30:01.510613 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510587 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-credential-socket\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.510723 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510619 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-istio-token\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.510723 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510658 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-envoy\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.510723 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510678 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxn78\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-kube-api-access-bxn78\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.510723 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-socket\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.510875 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510733 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-data\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.510875 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510769 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f75da6-d8eb-4147-a376-64f3f9939906-istio-podinfo\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.510875 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510787 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f75da6-d8eb-4147-a376-64f3f9939906-istiod-ca-cert\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.510875 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.510812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-certs\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f75da6-d8eb-4147-a376-64f3f9939906-istio-podinfo\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611583 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f75da6-d8eb-4147-a376-64f3f9939906-istiod-ca-cert\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-certs\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611653 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-credential-socket\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611674 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-istio-token\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611702 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-envoy\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611718 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxn78\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-kube-api-access-bxn78\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611770 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-socket\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.611826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.611818 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-data\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.612232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.612204 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-credential-socket\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.612374 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.612266 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-socket\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.612374 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.612330 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-data\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.612457 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.612385 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f75da6-d8eb-4147-a376-64f3f9939906-istiod-ca-cert\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.612457 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.612431 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-certs\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.614419 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.614398 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-envoy\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.614544 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.614528 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f75da6-d8eb-4147-a376-64f3f9939906-istio-podinfo\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.621007 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.620988 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-istio-token\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.621126 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.621103 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxn78\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-kube-api-access-bxn78\") pod \"e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.772544 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.772459 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:01.917450 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.917424 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk"] Jul 14 14:30:01.918365 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:30:01.918340 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f75da6_d8eb_4147_a376_64f3f9939906.slice/crio-1f8183f5c33cff7e6d70eb5706b0ca1ecf967d31564a64fed7c34033e6617474 WatchSource:0}: Error finding container 1f8183f5c33cff7e6d70eb5706b0ca1ecf967d31564a64fed7c34033e6617474: Status 404 returned error can't find the container with id 1f8183f5c33cff7e6d70eb5706b0ca1ecf967d31564a64fed7c34033e6617474 Jul 14 14:30:01.920480 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.920450 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:01.920550 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.920511 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:01.920594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:01.920550 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:02.436838 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:02.436795 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" event={"ID":"71f75da6-d8eb-4147-a376-64f3f9939906","Type":"ContainerStarted","Data":"316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a"} Jul 14 14:30:02.437302 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:02.436854 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" event={"ID":"71f75da6-d8eb-4147-a376-64f3f9939906","Type":"ContainerStarted","Data":"1f8183f5c33cff7e6d70eb5706b0ca1ecf967d31564a64fed7c34033e6617474"} Jul 14 14:30:02.464987 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:02.464929 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" podStartSLOduration=1.464912963 podStartE2EDuration="1.464912963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:30:02.461717613 +0000 UTC m=+2141.688400530" watchObservedRunningTime="2026-07-14 14:30:02.464912963 +0000 UTC m=+2141.691595881" Jul 14 14:30:02.555344 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:02.555320 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6" Jul 14 14:30:02.773116 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:02.773038 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:02.777512 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:02.777488 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:03.443546 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:03.443519 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6" Jul 14 14:30:03.443903 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:03.443544 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29733990-rz5x6" event={"ID":"9221b79f-5d96-4432-9fdf-7f3febf7ad6c","Type":"ContainerDied","Data":"f709c3696cb8063e8b16123b46e7db2a2a18a5f66d9de8b9c500f8d83766686c"} Jul 14 14:30:03.443903 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:03.443587 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f709c3696cb8063e8b16123b46e7db2a2a18a5f66d9de8b9c500f8d83766686c" Jul 14 14:30:03.443903 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:03.443828 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:03.444749 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:03.444735 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:09.739968 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.739928 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928"] Jul 14 14:30:09.740444 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.740308 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9221b79f-5d96-4432-9fdf-7f3febf7ad6c" containerName="cleanup" Jul 14 14:30:09.740444 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.740320 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9221b79f-5d96-4432-9fdf-7f3febf7ad6c" containerName="cleanup" Jul 14 14:30:09.740444 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.740394 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9221b79f-5d96-4432-9fdf-7f3febf7ad6c" containerName="cleanup" Jul 14 14:30:09.743726 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.743706 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.747793 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.747773 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-94a6fc-openshift-default-dockercfg-55str\"" Jul 14 14:30:09.759767 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.759743 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928"] Jul 14 14:30:09.885514 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885479 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-certs\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.885689 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885521 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-kube-api-access-qz4x5\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.885689 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885651 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-token\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.885783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istiod-ca-cert\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.885783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885714 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-podinfo\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.885783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-credential-socket\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.885783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-envoy\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.885980 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885857 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-socket\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.885980 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.885891 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-data\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986464 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986437 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-socket\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986464 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-data\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986679 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-certs\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986679 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986509 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-kube-api-access-qz4x5\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986679 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986554 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-token\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986679 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986575 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istiod-ca-cert\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986679 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986594 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-podinfo\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986679 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986614 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-credential-socket\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.986679 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986637 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-envoy\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.987034 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986856 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-socket\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.987034 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.986958 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-data\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.987034 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.987004 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-certs\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.987193 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.987142 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-credential-socket\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.987442 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.987421 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istiod-ca-cert\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.988851 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.988832 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-envoy\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.988977 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.988961 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-podinfo\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.996775 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.996722 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-token\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:09.997389 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:09.997363 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-kube-api-access-qz4x5\") pod \"e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:10.053901 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:10.053879 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:10.197562 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:10.197537 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928"] Jul 14 14:30:10.198306 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:30:10.198257 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod409b7ced_0db0_4859_a8ba_ac0bcaca817d.slice/crio-c8bdcd813ae1a438b269731650de1c5496383b2f2c7a0d22ec04c3b3ad940cc0 WatchSource:0}: Error finding container c8bdcd813ae1a438b269731650de1c5496383b2f2c7a0d22ec04c3b3ad940cc0: Status 404 returned error can't find the container with id c8bdcd813ae1a438b269731650de1c5496383b2f2c7a0d22ec04c3b3ad940cc0 Jul 14 14:30:10.200311 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:10.200267 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:10.200376 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:10.200343 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:10.200376 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:10.200371 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:10.482005 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:10.481970 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" event={"ID":"409b7ced-0db0-4859-a8ba-ac0bcaca817d","Type":"ContainerStarted","Data":"b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca"} Jul 14 14:30:10.482177 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:10.482032 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" event={"ID":"409b7ced-0db0-4859-a8ba-ac0bcaca817d","Type":"ContainerStarted","Data":"c8bdcd813ae1a438b269731650de1c5496383b2f2c7a0d22ec04c3b3ad940cc0"} Jul 14 14:30:10.506190 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:10.506099 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" podStartSLOduration=1.506084934 podStartE2EDuration="1.506084934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:30:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:30:10.503619981 +0000 UTC m=+2149.730302921" watchObservedRunningTime="2026-07-14 14:30:10.506084934 +0000 UTC m=+2149.732767908" Jul 14 14:30:11.054837 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:11.054803 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:11.059593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:11.059565 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:11.488162 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:11.488070 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:11.488821 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:11.488804 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:19.485471 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:19.485424 2564 patch_prober.go:28] interesting pod/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 14:30:19.485983 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:19.485506 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 14:30:20.551754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:20.551717 2564 generic.go:356] "Generic (PLEG): container finished" podID="71f75da6-d8eb-4147-a376-64f3f9939906" containerID="316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a" exitCode=0 Jul 14 14:30:20.552369 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:20.551775 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" event={"ID":"71f75da6-d8eb-4147-a376-64f3f9939906","Type":"ContainerDied","Data":"316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a"} Jul 14 14:30:20.552801 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:20.552786 2564 scope.go:117] "RemoveContainer" containerID="316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a" Jul 14 14:30:20.553966 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:20.553938 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:20.554040 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:20.554022 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:20.554087 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:20.554072 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:21.531182 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:21.531139 2564 scope.go:117] "RemoveContainer" containerID="84774d950ee9c57cfde36984f3acfb68b30689478a64e412dc77899273360817" Jul 14 14:30:21.579594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:21.579552 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" event={"ID":"71f75da6-d8eb-4147-a376-64f3f9939906","Type":"ContainerStarted","Data":"ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8"} Jul 14 14:30:21.773603 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:21.773565 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:21.773846 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:21.773816 2564 patch_prober.go:28] interesting pod/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 14 14:30:21.773908 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:21.773883 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 14 14:30:22.773340 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:22.773296 2564 patch_prober.go:28] interesting pod/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 14 14:30:22.773860 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:22.773380 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 14 14:30:23.773207 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:23.773171 2564 patch_prober.go:28] interesting pod/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 14 14:30:23.773388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:23.773234 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 14 14:30:24.773471 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:24.773426 2564 patch_prober.go:28] interesting pod/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 14 14:30:24.773870 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:24.773493 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 14 14:30:25.776160 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:25.776132 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:25.776660 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:25.776530 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:25.777219 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:25.777202 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:30:26.490199 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:26.490167 2564 patch_prober.go:28] interesting pod/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 14 14:30:26.490383 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:26.490221 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 14 14:30:26.610703 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:26.610672 2564 generic.go:356] "Generic (PLEG): container finished" podID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerID="b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca" exitCode=0 Jul 14 14:30:26.610858 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:26.610747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" event={"ID":"409b7ced-0db0-4859-a8ba-ac0bcaca817d","Type":"ContainerDied","Data":"b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca"} Jul 14 14:30:26.611227 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:26.611210 2564 scope.go:117] "RemoveContainer" containerID="b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca" Jul 14 14:30:26.612169 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:26.612144 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:26.612260 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:26.612209 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:26.612260 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:26.612241 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:30:27.617670 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:27.617637 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" event={"ID":"409b7ced-0db0-4859-a8ba-ac0bcaca817d","Type":"ContainerStarted","Data":"5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8"} Jul 14 14:30:28.054621 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:28.054590 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:28.054876 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:28.054850 2564 patch_prober.go:28] interesting pod/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 14 14:30:28.054927 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:28.054900 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 14 14:30:29.054373 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:29.054322 2564 patch_prober.go:28] interesting pod/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 14 14:30:29.054917 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:29.054399 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 14 14:30:30.054478 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:30.054439 2564 patch_prober.go:28] interesting pod/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 14 14:30:30.054961 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:30.054497 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 14 14:30:31.055123 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:31.055088 2564 patch_prober.go:28] interesting pod/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 14 14:30:31.055591 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:31.055132 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 14 14:30:32.054262 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:32.054224 2564 patch_prober.go:28] interesting pod/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 14 14:30:32.054469 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:32.054304 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 14 14:30:33.058216 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:33.058188 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:33.058683 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:33.058528 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:30:33.059149 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:30:33.059133 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:31:38.532808 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:38.532770 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk"] Jul 14 14:31:38.533309 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:38.533010 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" containerID="cri-o://ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8" gracePeriod=30 Jul 14 14:31:40.779253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:40.779209 2564 patch_prober.go:28] interesting pod/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 14:31:40.779647 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:40.779326 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 14:31:43.784362 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.784331 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:31:43.863860 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.863830 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f75da6-d8eb-4147-a376-64f3f9939906-istio-podinfo\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.863860 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.863865 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-certs\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.864107 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.863895 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f75da6-d8eb-4147-a376-64f3f9939906-istiod-ca-cert\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.864107 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.863927 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-credential-socket\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.864107 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.863964 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-envoy\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.864107 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864007 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxn78\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-kube-api-access-bxn78\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.864107 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864029 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-data\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.864107 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864055 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-istio-token\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.864445 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864129 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-socket\") pod \"71f75da6-d8eb-4147-a376-64f3f9939906\" (UID: \"71f75da6-d8eb-4147-a376-64f3f9939906\") " Jul 14 14:31:43.864445 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864130 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:31:43.864445 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864418 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:31:43.864612 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864431 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:31:43.864612 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864456 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71f75da6-d8eb-4147-a376-64f3f9939906-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:31:43.864717 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864588 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:43.864717 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864639 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71f75da6-d8eb-4147-a376-64f3f9939906-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:43.864717 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864653 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:43.864717 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.864667 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:43.865262 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.865235 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-data" (OuterVolumeSpecName: "istio-data") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:31:43.866099 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.866074 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71f75da6-d8eb-4147-a376-64f3f9939906-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:31:43.866212 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.866103 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:31:43.866556 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.866526 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-kube-api-access-bxn78" (OuterVolumeSpecName: "kube-api-access-bxn78") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "kube-api-access-bxn78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:31:43.866660 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.866537 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-istio-token" (OuterVolumeSpecName: "istio-token") pod "71f75da6-d8eb-4147-a376-64f3f9939906" (UID: "71f75da6-d8eb-4147-a376-64f3f9939906"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:31:43.965755 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.965720 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71f75da6-d8eb-4147-a376-64f3f9939906-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:43.965755 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.965748 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:43.965755 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.965758 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxn78\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-kube-api-access-bxn78\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:43.966071 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.965773 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71f75da6-d8eb-4147-a376-64f3f9939906-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:43.966071 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:43.965783 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71f75da6-d8eb-4147-a376-64f3f9939906-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:31:44.038683 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.038648 2564 generic.go:356] "Generic (PLEG): container finished" podID="71f75da6-d8eb-4147-a376-64f3f9939906" containerID="ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8" exitCode=0 Jul 14 14:31:44.038835 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.038691 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" Jul 14 14:31:44.038835 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.038725 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" event={"ID":"71f75da6-d8eb-4147-a376-64f3f9939906","Type":"ContainerDied","Data":"ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8"} Jul 14 14:31:44.038835 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.038773 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk" event={"ID":"71f75da6-d8eb-4147-a376-64f3f9939906","Type":"ContainerDied","Data":"1f8183f5c33cff7e6d70eb5706b0ca1ecf967d31564a64fed7c34033e6617474"} Jul 14 14:31:44.038835 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.038801 2564 scope.go:117] "RemoveContainer" containerID="ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8" Jul 14 14:31:44.048142 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.048121 2564 scope.go:117] "RemoveContainer" containerID="316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a" Jul 14 14:31:44.062231 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.062209 2564 scope.go:117] "RemoveContainer" containerID="ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8" Jul 14 14:31:44.062575 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:31:44.062546 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8\": container with ID starting with ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8 not found: ID does not exist" containerID="ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8" Jul 14 14:31:44.062672 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.062581 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8"} err="failed to get container status \"ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8\": rpc error: code = NotFound desc = could not find container \"ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8\": container with ID starting with ee5fb13f24a7ba1547f7ec5b6414af78b604826aae2e2eca2e3f9ec7b26952a8 not found: ID does not exist" Jul 14 14:31:44.062672 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.062601 2564 scope.go:117] "RemoveContainer" containerID="316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a" Jul 14 14:31:44.062907 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:31:44.062871 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a\": container with ID starting with 316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a not found: ID does not exist" containerID="316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a" Jul 14 14:31:44.063027 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.062900 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a"} err="failed to get container status \"316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a\": rpc error: code = NotFound desc = could not find container \"316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a\": container with ID starting with 316313ff72588d10610ce419a35b95a158c34435d5c9c3d78fe82b632ec77a0a not found: ID does not exist" Jul 14 14:31:44.066243 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.066217 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk"] Jul 14 14:31:44.072979 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:44.072948 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-0bd8ad-openshift-default-75dc978f95-6xwkk"] Jul 14 14:31:45.370625 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:31:45.370587 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" path="/var/lib/kubelet/pods/71f75da6-d8eb-4147-a376-64f3f9939906/volumes" Jul 14 14:32:04.835749 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:04.835713 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928"] Jul 14 14:32:04.836212 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:04.836002 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" containerID="cri-o://5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8" gracePeriod=30 Jul 14 14:32:05.953052 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.953018 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8"] Jul 14 14:32:05.953459 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.953402 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" Jul 14 14:32:05.953459 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.953414 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" Jul 14 14:32:05.953459 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.953436 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" Jul 14 14:32:05.953459 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.953443 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" Jul 14 14:32:05.953595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.953500 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" Jul 14 14:32:05.953595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.953508 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="71f75da6-d8eb-4147-a376-64f3f9939906" containerName="istio-proxy" Jul 14 14:32:05.956496 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.956475 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.959310 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.959272 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a7298c-openshift-default-dockercfg-z68dp\"" Jul 14 14:32:05.969121 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-data\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.969253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-podinfo\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.969253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-certs\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.969400 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-credential-socket\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.969400 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969308 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-envoy\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.969400 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969388 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-token\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.969557 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969426 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-socket\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.969557 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969474 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h74x\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-kube-api-access-9h74x\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.969557 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.969517 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istiod-ca-cert\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:05.971629 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:05.971608 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8"] Jul 14 14:32:06.070214 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070171 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-token\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.070436 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070227 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-socket\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.070436 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070395 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h74x\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-kube-api-access-9h74x\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.070595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070438 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istiod-ca-cert\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.070595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070488 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-data\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.070595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070526 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-podinfo\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.070595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070579 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-certs\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.070815 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-credential-socket\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.070815 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.070659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-envoy\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.071652 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.071439 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istiod-ca-cert\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.071652 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.071526 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-certs\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.071858 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.071662 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-socket\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.071918 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.071905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-data\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.072173 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.072131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-credential-socket\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.074095 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.074070 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-envoy\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.074624 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.074604 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-podinfo\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.078953 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.078928 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-token\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.079078 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.079059 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h74x\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-kube-api-access-9h74x\") pod \"e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.266261 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.266175 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:06.419145 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.419111 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8"] Jul 14 14:32:06.420410 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:32:06.420383 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b13153c_06ac_4d63_a5ef_aa2547ba7ce1.slice/crio-e7d4589af7bb8f80fdbb8acf36324916367f31a9943f1d36e579a56f13c628cb WatchSource:0}: Error finding container e7d4589af7bb8f80fdbb8acf36324916367f31a9943f1d36e579a56f13c628cb: Status 404 returned error can't find the container with id e7d4589af7bb8f80fdbb8acf36324916367f31a9943f1d36e579a56f13c628cb Jul 14 14:32:06.422499 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.422465 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:06.422567 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.422533 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:06.422605 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:06.422563 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:07.162901 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:07.162858 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" event={"ID":"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1","Type":"ContainerStarted","Data":"88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890"} Jul 14 14:32:07.163328 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:07.162922 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" event={"ID":"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1","Type":"ContainerStarted","Data":"e7d4589af7bb8f80fdbb8acf36324916367f31a9943f1d36e579a56f13c628cb"} Jul 14 14:32:07.187743 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:07.187698 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" podStartSLOduration=2.187680419 podStartE2EDuration="2.187680419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:32:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:32:07.186039792 +0000 UTC m=+2266.412722731" watchObservedRunningTime="2026-07-14 14:32:07.187680419 +0000 UTC m=+2266.414363340" Jul 14 14:32:07.266334 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:07.266300 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:07.271595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:07.271571 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:08.168410 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:08.168381 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:08.169390 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:08.169371 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:10.088449 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.088425 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:32:10.109433 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109364 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-credential-socket\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109433 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109425 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-certs\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109612 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109492 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-token\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109612 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109549 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istiod-ca-cert\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109612 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109578 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-podinfo\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109726 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109629 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-envoy\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109726 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109645 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:32:10.109726 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109654 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-kube-api-access-qz4x5\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109726 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109679 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:32:10.109883 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109727 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-socket\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109883 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109807 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-data\") pod \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\" (UID: \"409b7ced-0db0-4859-a8ba-ac0bcaca817d\") " Jul 14 14:32:10.109997 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.109976 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:32:10.110045 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.110002 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:32:10.110371 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.110231 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.110371 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.110258 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.110371 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.110273 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.110371 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.110311 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.112039 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.110740 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-data" (OuterVolumeSpecName: "istio-data") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:32:10.113818 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.113665 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:32:10.114112 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.114011 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-token" (OuterVolumeSpecName: "istio-token") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:32:10.114112 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.114085 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-kube-api-access-qz4x5" (OuterVolumeSpecName: "kube-api-access-qz4x5") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "kube-api-access-qz4x5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:32:10.114112 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.114080 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "409b7ced-0db0-4859-a8ba-ac0bcaca817d" (UID: "409b7ced-0db0-4859-a8ba-ac0bcaca817d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:32:10.180078 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.180037 2564 generic.go:356] "Generic (PLEG): container finished" podID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerID="5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8" exitCode=0 Jul 14 14:32:10.180248 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.180088 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" Jul 14 14:32:10.180248 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.180115 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" event={"ID":"409b7ced-0db0-4859-a8ba-ac0bcaca817d","Type":"ContainerDied","Data":"5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8"} Jul 14 14:32:10.180248 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.180165 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928" event={"ID":"409b7ced-0db0-4859-a8ba-ac0bcaca817d","Type":"ContainerDied","Data":"c8bdcd813ae1a438b269731650de1c5496383b2f2c7a0d22ec04c3b3ad940cc0"} Jul 14 14:32:10.180248 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.180181 2564 scope.go:117] "RemoveContainer" containerID="5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8" Jul 14 14:32:10.190390 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.190373 2564 scope.go:117] "RemoveContainer" containerID="b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca" Jul 14 14:32:10.201448 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.201425 2564 scope.go:117] "RemoveContainer" containerID="5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8" Jul 14 14:32:10.201950 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:32:10.201787 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8\": container with ID starting with 5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8 not found: ID does not exist" containerID="5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8" Jul 14 14:32:10.201950 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.201835 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8"} err="failed to get container status \"5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8\": rpc error: code = NotFound desc = could not find container \"5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8\": container with ID starting with 5fada6a106ab8ddc7a1f5259156b3a9f446cb5103941b0c4d67aac372fd6eab8 not found: ID does not exist" Jul 14 14:32:10.201950 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.201863 2564 scope.go:117] "RemoveContainer" containerID="b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca" Jul 14 14:32:10.202204 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:32:10.202161 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca\": container with ID starting with b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca not found: ID does not exist" containerID="b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca" Jul 14 14:32:10.202204 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.202189 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca"} err="failed to get container status \"b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca\": rpc error: code = NotFound desc = could not find container \"b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca\": container with ID starting with b420fc4f99465fb29c2449257c726b6b047711fb2ad8feb09ca6c356442594ca not found: ID does not exist" Jul 14 14:32:10.207302 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.207255 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928"] Jul 14 14:32:10.211232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.210923 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.211232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.210949 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.211232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.210961 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.211232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.210976 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz4x5\" (UniqueName: \"kubernetes.io/projected/409b7ced-0db0-4859-a8ba-ac0bcaca817d-kube-api-access-qz4x5\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.211232 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.210988 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/409b7ced-0db0-4859-a8ba-ac0bcaca817d-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:32:10.213989 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:10.213963 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-94a6fc-openshift-default-7b46d79c5c-62928"] Jul 14 14:32:11.369990 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:11.369956 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" path="/var/lib/kubelet/pods/409b7ced-0db0-4859-a8ba-ac0bcaca817d/volumes" Jul 14 14:32:14.758654 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.758621 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx"] Jul 14 14:32:14.759258 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.759062 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" Jul 14 14:32:14.759258 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.759074 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" Jul 14 14:32:14.759258 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.759140 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" Jul 14 14:32:14.759258 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.759147 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" Jul 14 14:32:14.763471 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.763451 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.766633 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.766613 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-9343ac-openshift-default-dockercfg-jftqp\"" Jul 14 14:32:14.776772 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.776751 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx"] Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.852952 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-data\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.853051 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-socket\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.853095 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwxj6\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-kube-api-access-qwxj6\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.853127 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-credential-socket\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.853168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-istio-token\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.853238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a98034-cc56-46b1-97cf-306221f5f233-istio-podinfo\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.853260 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a98034-cc56-46b1-97cf-306221f5f233-istiod-ca-cert\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.853332 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-envoy\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.853483 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.853363 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-certs\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954039 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.953996 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-credential-socket\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954039 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954044 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-istio-token\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954264 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a98034-cc56-46b1-97cf-306221f5f233-istio-podinfo\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954264 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954097 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a98034-cc56-46b1-97cf-306221f5f233-istiod-ca-cert\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954264 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954122 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-envoy\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954264 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954140 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-certs\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954264 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954177 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-data\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954264 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954223 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-socket\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954264 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954255 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwxj6\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-kube-api-access-qwxj6\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954761 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954711 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-credential-socket\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954901 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954808 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-socket\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954963 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954907 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-certs\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.954963 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.954946 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-data\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.955062 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.955042 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a98034-cc56-46b1-97cf-306221f5f233-istiod-ca-cert\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.956710 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.956691 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-envoy\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.956955 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.956934 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a98034-cc56-46b1-97cf-306221f5f233-istio-podinfo\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.962409 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.962389 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-istio-token\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:14.962683 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:14.962665 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwxj6\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-kube-api-access-qwxj6\") pod \"e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:15.074569 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:15.074532 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:15.251426 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:15.251394 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx"] Jul 14 14:32:15.253598 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:32:15.253566 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76a98034_cc56_46b1_97cf_306221f5f233.slice/crio-6cb55ed699d1183facd716f27170048d23565140bf05d0475c4406a97e074f83 WatchSource:0}: Error finding container 6cb55ed699d1183facd716f27170048d23565140bf05d0475c4406a97e074f83: Status 404 returned error can't find the container with id 6cb55ed699d1183facd716f27170048d23565140bf05d0475c4406a97e074f83 Jul 14 14:32:15.255779 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:15.255737 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:15.255886 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:15.255812 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:15.255886 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:15.255851 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:16.215129 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:16.215095 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" event={"ID":"76a98034-cc56-46b1-97cf-306221f5f233","Type":"ContainerStarted","Data":"538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0"} Jul 14 14:32:16.215535 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:16.215144 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" event={"ID":"76a98034-cc56-46b1-97cf-306221f5f233","Type":"ContainerStarted","Data":"6cb55ed699d1183facd716f27170048d23565140bf05d0475c4406a97e074f83"} Jul 14 14:32:17.074869 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:17.074831 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:17.079809 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:17.079786 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:17.116343 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:17.116301 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podStartSLOduration=3.116265816 podStartE2EDuration="3.116265816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:32:16.241783891 +0000 UTC m=+2275.468466810" watchObservedRunningTime="2026-07-14 14:32:17.116265816 +0000 UTC m=+2276.342948733" Jul 14 14:32:17.220585 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:17.220560 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:17.221410 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:17.221385 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:21.244948 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:21.244873 2564 generic.go:356] "Generic (PLEG): container finished" podID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerID="88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890" exitCode=0 Jul 14 14:32:21.245350 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:21.244949 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" event={"ID":"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1","Type":"ContainerDied","Data":"88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890"} Jul 14 14:32:21.245399 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:21.245378 2564 scope.go:117] "RemoveContainer" containerID="88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890" Jul 14 14:32:21.246364 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:21.246336 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:21.246436 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:21.246418 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:21.246506 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:21.246491 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:22.253683 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:22.253635 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" event={"ID":"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1","Type":"ContainerStarted","Data":"d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996"} Jul 14 14:32:22.267150 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:22.267116 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:22.268388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:22.268354 2564 patch_prober.go:28] interesting pod/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 14 14:32:22.268553 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:22.268430 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 14 14:32:23.168617 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:23.168576 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:23.267462 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:23.267427 2564 patch_prober.go:28] interesting pod/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 14 14:32:23.267829 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:23.267486 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 14 14:32:24.266780 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:24.266748 2564 patch_prober.go:28] interesting pod/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 14 14:32:24.266959 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:24.266798 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 14 14:32:25.267051 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:25.267026 2564 patch_prober.go:28] interesting pod/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 14 14:32:25.267462 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:25.267074 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 14 14:32:26.267159 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:26.267120 2564 patch_prober.go:28] interesting pod/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body= Jul 14 14:32:26.267581 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:26.267179 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" Jul 14 14:32:27.270564 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:27.270531 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:27.271537 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:27.271514 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:32:33.262501 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:33.262449 2564 patch_prober.go:28] interesting pod/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 14:32:33.263023 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:33.262546 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 14:32:35.330677 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:35.330642 2564 generic.go:356] "Generic (PLEG): container finished" podID="76a98034-cc56-46b1-97cf-306221f5f233" containerID="538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0" exitCode=0 Jul 14 14:32:35.331093 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:35.330716 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" event={"ID":"76a98034-cc56-46b1-97cf-306221f5f233","Type":"ContainerDied","Data":"538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0"} Jul 14 14:32:35.331180 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:35.331162 2564 scope.go:117] "RemoveContainer" containerID="538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0" Jul 14 14:32:35.332192 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:35.332160 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:35.332306 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:35.332228 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:35.332306 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:35.332259 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:32:36.338049 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:36.338017 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" event={"ID":"76a98034-cc56-46b1-97cf-306221f5f233","Type":"ContainerStarted","Data":"5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18"} Jul 14 14:32:37.074923 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:37.074890 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:37.075104 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:37.075069 2564 patch_prober.go:28] interesting pod/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 14 14:32:37.075146 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:37.075116 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 14 14:32:38.075586 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:38.075544 2564 patch_prober.go:28] interesting pod/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 14 14:32:38.076030 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:38.075602 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 14 14:32:39.074913 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:39.074879 2564 patch_prober.go:28] interesting pod/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 14 14:32:39.075095 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:39.074934 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 14 14:32:40.074992 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:40.074955 2564 patch_prober.go:28] interesting pod/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 14 14:32:40.075420 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:40.075026 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 14 14:32:41.075151 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:41.075115 2564 patch_prober.go:28] interesting pod/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 14 14:32:41.075617 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:41.075185 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 14 14:32:42.078943 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:42.078911 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:42.079421 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:42.079178 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:42.079816 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:42.079801 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:32:57.415004 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:57.414973 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8"] Jul 14 14:32:57.415579 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:32:57.415169 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" containerID="cri-o://d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996" gracePeriod=30 Jul 14 14:33:02.662318 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.662293 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:33:02.727817 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.727719 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-envoy\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.727817 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.727768 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istiod-ca-cert\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.727817 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.727797 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-socket\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.728105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.727838 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h74x\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-kube-api-access-9h74x\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.728105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.727879 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-certs\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.728105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.727910 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-podinfo\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.728105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.727942 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-credential-socket\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.728105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.727976 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-token\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.728105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.728010 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-data\") pod \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\" (UID: \"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1\") " Jul 14 14:33:02.728448 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.728181 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:02.728448 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.728199 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:02.728448 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.728319 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:33:02.728448 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.728400 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:02.728448 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.728410 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:02.728448 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.728423 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:02.728972 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.728940 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:02.729948 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.729925 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:02.730065 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.730040 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-kube-api-access-9h74x" (OuterVolumeSpecName: "kube-api-access-9h74x") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "kube-api-access-9h74x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:33:02.730217 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.730193 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:33:02.730337 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.730311 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" (UID: "8b13153c-06ac-4d63-a5ef-aa2547ba7ce1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:33:02.829523 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.829474 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:02.829523 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.829515 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:02.829523 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.829525 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:02.829523 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.829533 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:02.829523 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.829542 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:02.829903 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.829550 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:02.829903 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:02.829558 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h74x\" (UniqueName: \"kubernetes.io/projected/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1-kube-api-access-9h74x\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:03.498270 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.498231 2564 generic.go:356] "Generic (PLEG): container finished" podID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerID="d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996" exitCode=0 Jul 14 14:33:03.498476 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.498315 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" Jul 14 14:33:03.498476 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.498315 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" event={"ID":"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1","Type":"ContainerDied","Data":"d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996"} Jul 14 14:33:03.498476 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.498402 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8" event={"ID":"8b13153c-06ac-4d63-a5ef-aa2547ba7ce1","Type":"ContainerDied","Data":"e7d4589af7bb8f80fdbb8acf36324916367f31a9943f1d36e579a56f13c628cb"} Jul 14 14:33:03.498476 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.498434 2564 scope.go:117] "RemoveContainer" containerID="d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996" Jul 14 14:33:03.507497 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.507479 2564 scope.go:117] "RemoveContainer" containerID="88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890" Jul 14 14:33:03.515808 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.515789 2564 scope.go:117] "RemoveContainer" containerID="d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996" Jul 14 14:33:03.516041 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:33:03.516021 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996\": container with ID starting with d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996 not found: ID does not exist" containerID="d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996" Jul 14 14:33:03.516092 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.516050 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996"} err="failed to get container status \"d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996\": rpc error: code = NotFound desc = could not find container \"d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996\": container with ID starting with d14afd224fa754c8f0388718160d533f68e2685af86693adcafbc25130e2a996 not found: ID does not exist" Jul 14 14:33:03.516092 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.516067 2564 scope.go:117] "RemoveContainer" containerID="88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890" Jul 14 14:33:03.516325 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:33:03.516311 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890\": container with ID starting with 88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890 not found: ID does not exist" containerID="88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890" Jul 14 14:33:03.516388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.516329 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890"} err="failed to get container status \"88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890\": rpc error: code = NotFound desc = could not find container \"88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890\": container with ID starting with 88c474deebbb674fe30142ea25d228b48f66b8d34972fc6ee3691bc7de138890 not found: ID does not exist" Jul 14 14:33:03.531447 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.531410 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8"] Jul 14 14:33:03.537772 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:03.537750 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a7298c-openshift-default-765b7bb875-xt4v8"] Jul 14 14:33:05.369786 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:05.369749 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" path="/var/lib/kubelet/pods/8b13153c-06ac-4d63-a5ef-aa2547ba7ce1/volumes" Jul 14 14:33:23.700925 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:23.700894 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx"] Jul 14 14:33:23.701336 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:23.701155 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" containerID="cri-o://5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18" gracePeriod=30 Jul 14 14:33:24.688748 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.688714 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk"] Jul 14 14:33:24.689167 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.689151 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" Jul 14 14:33:24.689316 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.689170 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" Jul 14 14:33:24.689316 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.689194 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" Jul 14 14:33:24.689316 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.689200 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="409b7ced-0db0-4859-a8ba-ac0bcaca817d" containerName="istio-proxy" Jul 14 14:33:24.689316 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.689261 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" Jul 14 14:33:24.689316 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.689272 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" Jul 14 14:33:24.692721 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.692706 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.695171 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.695151 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8442f2-openshift-default-dockercfg-tj4ds\"" Jul 14 14:33:24.706315 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.706275 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk"] Jul 14 14:33:24.845568 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-data\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.845747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845582 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-credential-socket\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.845747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845604 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-envoy\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.845747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845670 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-socket\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.845747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845706 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-certs\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.845747 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845745 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86gjb\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-kube-api-access-86gjb\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.845994 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845831 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-istio-token\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.845994 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845878 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3666793e-53d0-48cf-b350-f6b3890dda86-istio-podinfo\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.845994 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.845942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3666793e-53d0-48cf-b350-f6b3890dda86-istiod-ca-cert\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947319 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947219 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-istio-token\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947319 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947260 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3666793e-53d0-48cf-b350-f6b3890dda86-istio-podinfo\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947319 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947316 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3666793e-53d0-48cf-b350-f6b3890dda86-istiod-ca-cert\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-data\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947399 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-credential-socket\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-envoy\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-socket\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947488 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-certs\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86gjb\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-kube-api-access-86gjb\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947918 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947894 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-certs\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.947981 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.947954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-data\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.948072 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.948054 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3666793e-53d0-48cf-b350-f6b3890dda86-istiod-ca-cert\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.948222 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.948202 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-socket\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.948308 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.948227 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-credential-socket\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.949847 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.949826 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3666793e-53d0-48cf-b350-f6b3890dda86-istio-podinfo\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.950118 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.950098 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-envoy\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.955206 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.955173 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-istio-token\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:24.955365 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:24.955348 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86gjb\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-kube-api-access-86gjb\") pod \"e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:25.001954 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:25.001928 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:25.136730 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:25.136700 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk"] Jul 14 14:33:25.138012 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:33:25.137985 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3666793e_53d0_48cf_b350_f6b3890dda86.slice/crio-ee4d9d798284d3662f968a7dd75014d0ee67792013e86e914a4e0500d0fa5f5f WatchSource:0}: Error finding container ee4d9d798284d3662f968a7dd75014d0ee67792013e86e914a4e0500d0fa5f5f: Status 404 returned error can't find the container with id ee4d9d798284d3662f968a7dd75014d0ee67792013e86e914a4e0500d0fa5f5f Jul 14 14:33:25.140263 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:25.140230 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:25.140393 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:25.140315 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:25.140393 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:25.140352 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:25.616935 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:25.616893 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" event={"ID":"3666793e-53d0-48cf-b350-f6b3890dda86","Type":"ContainerStarted","Data":"8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3"} Jul 14 14:33:25.617120 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:25.616962 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" event={"ID":"3666793e-53d0-48cf-b350-f6b3890dda86","Type":"ContainerStarted","Data":"ee4d9d798284d3662f968a7dd75014d0ee67792013e86e914a4e0500d0fa5f5f"} Jul 14 14:33:25.638835 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:25.638789 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" podStartSLOduration=1.6387741999999998 podStartE2EDuration="1.6387742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:33:25.636919532 +0000 UTC m=+2344.863602452" watchObservedRunningTime="2026-07-14 14:33:25.6387742 +0000 UTC m=+2344.865457117" Jul 14 14:33:26.002243 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:26.002163 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:26.006985 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:26.006958 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:26.623412 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:26.623380 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:26.624209 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:26.624191 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:27.081428 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:27.081392 2564 patch_prober.go:28] interesting pod/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 14:33:27.081819 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:27.081455 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 14:33:28.956072 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.956050 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:33:28.984146 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984064 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-credential-socket\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984146 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984105 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-istio-token\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984146 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984132 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-socket\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984435 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984197 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-envoy\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984435 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984235 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a98034-cc56-46b1-97cf-306221f5f233-istiod-ca-cert\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984435 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984258 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-data\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984435 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984327 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-certs\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984435 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984353 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwxj6\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-kube-api-access-qwxj6\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984435 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984368 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:28.984435 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984402 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a98034-cc56-46b1-97cf-306221f5f233-istio-podinfo\") pod \"76a98034-cc56-46b1-97cf-306221f5f233\" (UID: \"76a98034-cc56-46b1-97cf-306221f5f233\") " Jul 14 14:33:28.984780 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984583 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:28.984780 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984714 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:28.984897 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984793 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:28.984897 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984829 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:28.984897 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984844 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:28.985062 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.984956 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76a98034-cc56-46b1-97cf-306221f5f233-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:33:28.985634 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.985597 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-data" (OuterVolumeSpecName: "istio-data") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:28.987216 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.987183 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76a98034-cc56-46b1-97cf-306221f5f233-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:33:28.987216 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.987196 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-kube-api-access-qwxj6" (OuterVolumeSpecName: "kube-api-access-qwxj6") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "kube-api-access-qwxj6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:33:28.987368 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.987245 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:33:28.987368 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:28.987250 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-istio-token" (OuterVolumeSpecName: "istio-token") pod "76a98034-cc56-46b1-97cf-306221f5f233" (UID: "76a98034-cc56-46b1-97cf-306221f5f233"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:33:29.085685 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.085648 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:29.085685 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.085678 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:29.085685 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.085687 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a98034-cc56-46b1-97cf-306221f5f233-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:29.085685 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.085694 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a98034-cc56-46b1-97cf-306221f5f233-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:29.085972 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.085702 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwxj6\" (UniqueName: \"kubernetes.io/projected/76a98034-cc56-46b1-97cf-306221f5f233-kube-api-access-qwxj6\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:29.085972 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.085713 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a98034-cc56-46b1-97cf-306221f5f233-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:33:29.641099 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.641016 2564 generic.go:356] "Generic (PLEG): container finished" podID="76a98034-cc56-46b1-97cf-306221f5f233" containerID="5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18" exitCode=0 Jul 14 14:33:29.641099 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.641068 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" Jul 14 14:33:29.641099 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.641086 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" event={"ID":"76a98034-cc56-46b1-97cf-306221f5f233","Type":"ContainerDied","Data":"5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18"} Jul 14 14:33:29.641373 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.641128 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx" event={"ID":"76a98034-cc56-46b1-97cf-306221f5f233","Type":"ContainerDied","Data":"6cb55ed699d1183facd716f27170048d23565140bf05d0475c4406a97e074f83"} Jul 14 14:33:29.641373 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.641148 2564 scope.go:117] "RemoveContainer" containerID="5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18" Jul 14 14:33:29.650115 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.650098 2564 scope.go:117] "RemoveContainer" containerID="538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0" Jul 14 14:33:29.659136 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.659114 2564 scope.go:117] "RemoveContainer" containerID="5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18" Jul 14 14:33:29.659521 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:33:29.659501 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18\": container with ID starting with 5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18 not found: ID does not exist" containerID="5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18" Jul 14 14:33:29.659595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.659540 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18"} err="failed to get container status \"5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18\": rpc error: code = NotFound desc = could not find container \"5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18\": container with ID starting with 5388ce4e294ee60cb459e8fa5930a026cab354cb243da107722a919944788b18 not found: ID does not exist" Jul 14 14:33:29.659595 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.659558 2564 scope.go:117] "RemoveContainer" containerID="538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0" Jul 14 14:33:29.659825 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:33:29.659809 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0\": container with ID starting with 538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0 not found: ID does not exist" containerID="538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0" Jul 14 14:33:29.659870 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.659830 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0"} err="failed to get container status \"538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0\": rpc error: code = NotFound desc = could not find container \"538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0\": container with ID starting with 538713b9bc4634ad1c1b0ce8e0382e0bc13bd8de4d952dbf15cd86766e535ed0 not found: ID does not exist" Jul 14 14:33:29.660593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.660574 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx"] Jul 14 14:33:29.667313 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:29.667272 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-9343ac-openshift-default-688ffcfb67-zzwlx"] Jul 14 14:33:31.370762 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:31.370729 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76a98034-cc56-46b1-97cf-306221f5f233" path="/var/lib/kubelet/pods/76a98034-cc56-46b1-97cf-306221f5f233/volumes" Jul 14 14:33:33.003259 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003221 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4"] Jul 14 14:33:33.003754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003620 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" Jul 14 14:33:33.003754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003633 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" Jul 14 14:33:33.003754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003647 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" Jul 14 14:33:33.003754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003654 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" Jul 14 14:33:33.003754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003667 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" Jul 14 14:33:33.003754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003674 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b13153c-06ac-4d63-a5ef-aa2547ba7ce1" containerName="istio-proxy" Jul 14 14:33:33.003754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003733 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" Jul 14 14:33:33.003754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.003743 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="76a98034-cc56-46b1-97cf-306221f5f233" containerName="istio-proxy" Jul 14 14:33:33.008375 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.008354 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.010916 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.010895 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f8f81e-openshift-default-dockercfg-ftwbk\"" Jul 14 14:33:33.017716 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.017695 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4"] Jul 14 14:33:33.021155 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021134 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-data\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.021272 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021180 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-token\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.021272 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-podinfo\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.021272 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021246 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b848s\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-kube-api-access-b848s\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.021478 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-credential-socket\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.021478 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021325 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-certs\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.021478 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istiod-ca-cert\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.021478 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021446 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-socket\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.021693 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.021486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-envoy\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122470 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122436 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-socket\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122644 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-envoy\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122644 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-data\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122644 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122576 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-token\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122644 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-podinfo\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122644 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122634 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b848s\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-kube-api-access-b848s\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122677 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-credential-socket\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-certs\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122761 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istiod-ca-cert\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122833 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-socket\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.122936 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.122901 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-data\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.123204 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.123096 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-credential-socket\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.123204 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.123131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-certs\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.123461 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.123441 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istiod-ca-cert\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.124757 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.124738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-envoy\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.124879 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.124862 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-podinfo\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.132824 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.132792 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-token\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.132992 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.132962 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b848s\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-kube-api-access-b848s\") pod \"e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.320360 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.320314 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:33.461754 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.461720 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4"] Jul 14 14:33:33.462766 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:33:33.462736 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd5dfd1_2ebf_4689_b79a_5f706e2a63fb.slice/crio-07a1f775f9e8b4afa01886bcedb54554e255a891f9d57a77e0323f292e8ba33c WatchSource:0}: Error finding container 07a1f775f9e8b4afa01886bcedb54554e255a891f9d57a77e0323f292e8ba33c: Status 404 returned error can't find the container with id 07a1f775f9e8b4afa01886bcedb54554e255a891f9d57a77e0323f292e8ba33c Jul 14 14:33:33.464833 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.464796 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:33.464887 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.464861 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:33.464927 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.464891 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:33.664888 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.664789 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" event={"ID":"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb","Type":"ContainerStarted","Data":"501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62"} Jul 14 14:33:33.664888 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.664855 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" event={"ID":"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb","Type":"ContainerStarted","Data":"07a1f775f9e8b4afa01886bcedb54554e255a891f9d57a77e0323f292e8ba33c"} Jul 14 14:33:33.688000 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:33.687939 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" podStartSLOduration=1.687906392 podStartE2EDuration="1.687906392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:33:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:33:33.686097723 +0000 UTC m=+2352.912780643" watchObservedRunningTime="2026-07-14 14:33:33.687906392 +0000 UTC m=+2352.914589313" Jul 14 14:33:34.320583 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:34.320539 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:34.333806 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:34.333760 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:34.670591 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:34.670507 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:34.671438 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:34.671419 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:40.709807 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:40.709772 2564 generic.go:356] "Generic (PLEG): container finished" podID="3666793e-53d0-48cf-b350-f6b3890dda86" containerID="8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3" exitCode=0 Jul 14 14:33:40.710139 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:40.709845 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" event={"ID":"3666793e-53d0-48cf-b350-f6b3890dda86","Type":"ContainerDied","Data":"8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3"} Jul 14 14:33:40.710334 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:40.710317 2564 scope.go:117] "RemoveContainer" containerID="8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3" Jul 14 14:33:40.711266 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:40.711241 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:40.711424 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:40.711330 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:40.711424 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:40.711387 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:41.624188 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:41.624114 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:41.717710 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:41.717671 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" event={"ID":"3666793e-53d0-48cf-b350-f6b3890dda86","Type":"ContainerStarted","Data":"98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818"} Jul 14 14:33:42.002332 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:42.002212 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:42.003491 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:42.003465 2564 patch_prober.go:28] interesting pod/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 14:33:42.003608 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:42.003519 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 14:33:43.002886 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:43.002851 2564 patch_prober.go:28] interesting pod/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 14:33:43.003328 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:43.002907 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 14:33:44.002663 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:44.002613 2564 patch_prober.go:28] interesting pod/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 14:33:44.002881 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:44.002677 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 14:33:45.003428 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:45.003385 2564 patch_prober.go:28] interesting pod/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 14:33:45.003826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:45.003456 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 14:33:46.002834 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:46.002801 2564 patch_prober.go:28] interesting pod/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 14:33:46.003010 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:46.002865 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 14:33:47.003030 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:47.002991 2564 patch_prober.go:28] interesting pod/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 14:33:47.003601 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:47.003054 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 14:33:47.763584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:47.763491 2564 generic.go:356] "Generic (PLEG): container finished" podID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerID="501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62" exitCode=0 Jul 14 14:33:47.763584 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:47.763564 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" event={"ID":"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb","Type":"ContainerDied","Data":"501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62"} Jul 14 14:33:47.763969 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:47.763956 2564 scope.go:117] "RemoveContainer" containerID="501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62" Jul 14 14:33:47.765062 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:47.765035 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:47.765124 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:47.765108 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:47.765159 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:47.765148 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 14 14:33:48.007437 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:48.007408 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:48.007895 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:48.007780 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:48.008614 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:48.008592 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:33:48.772491 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:48.772453 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" event={"ID":"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb","Type":"ContainerStarted","Data":"2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a"} Jul 14 14:33:49.320541 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:49.320509 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:49.321983 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:49.321940 2564 patch_prober.go:28] interesting pod/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 14 14:33:49.322123 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:49.322040 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 14 14:33:49.671334 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:49.671219 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:50.321221 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:50.321185 2564 patch_prober.go:28] interesting pod/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 14 14:33:50.321728 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:50.321251 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 14 14:33:51.321228 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:51.321198 2564 patch_prober.go:28] interesting pod/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 14 14:33:51.321788 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:51.321256 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 14 14:33:52.320636 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:52.320603 2564 patch_prober.go:28] interesting pod/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 14 14:33:52.320827 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:52.320656 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 14 14:33:53.324637 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:53.324606 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:33:53.325616 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:33:53.325597 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:34:21.480588 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:34:21.480557 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:34:21.485342 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:34:21.485321 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:37:45.094412 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.094328 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk"] Jul 14 14:37:45.097265 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.097244 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.101194 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.101167 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"openshift-service-ca.crt\"" Jul 14 14:37:45.101194 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.101168 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"kube-root-ca.crt\"" Jul 14 14:37:45.101402 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.101168 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"default-dockercfg-n64l6\"" Jul 14 14:37:45.101402 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.101228 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8442f2\"/\"rate-test-model-8442f2-kserve-self-signed-certs\"" Jul 14 14:37:45.108223 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.108199 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk"] Jul 14 14:37:45.156445 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.156406 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-tmp-dir\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.156445 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.156447 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-model-cache\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.156658 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.156468 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-dshm\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.156658 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.156519 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-home\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.156658 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.156585 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c935dec8-22be-4daf-a60b-fb6087cd1a92-tls-certs\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.156658 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.156616 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbf7z\" (UniqueName: \"kubernetes.io/projected/c935dec8-22be-4daf-a60b-fb6087cd1a92-kube-api-access-dbf7z\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.257358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.257237 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c935dec8-22be-4daf-a60b-fb6087cd1a92-tls-certs\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.257358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.257326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbf7z\" (UniqueName: \"kubernetes.io/projected/c935dec8-22be-4daf-a60b-fb6087cd1a92-kube-api-access-dbf7z\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.257653 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.257441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-tmp-dir\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.257653 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.257471 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-model-cache\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.257653 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.257504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-dshm\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.257653 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.257548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-home\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.258061 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.258033 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-model-cache\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.258181 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.258070 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-tmp-dir\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.258430 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.258381 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-home\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.261246 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.261223 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c935dec8-22be-4daf-a60b-fb6087cd1a92-tls-certs\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.261484 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.261465 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-dshm\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.270381 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.270358 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbf7z\" (UniqueName: \"kubernetes.io/projected/c935dec8-22be-4daf-a60b-fb6087cd1a92-kube-api-access-dbf7z\") pod \"rate-test-model-8442f2-kserve-7c5549488c-wtfmk\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.408120 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.408027 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:45.542122 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:45.542092 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk"] Jul 14 14:37:45.543605 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:37:45.543575 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc935dec8_22be_4daf_a60b_fb6087cd1a92.slice/crio-4c6ced132188911f0a13374b6fe38e3fe560a374c5335ec7d727a7a58ca1d258 WatchSource:0}: Error finding container 4c6ced132188911f0a13374b6fe38e3fe560a374c5335ec7d727a7a58ca1d258: Status 404 returned error can't find the container with id 4c6ced132188911f0a13374b6fe38e3fe560a374c5335ec7d727a7a58ca1d258 Jul 14 14:37:46.075617 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:46.075582 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" event={"ID":"c935dec8-22be-4daf-a60b-fb6087cd1a92","Type":"ContainerStarted","Data":"291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea"} Jul 14 14:37:46.075828 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:46.075633 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" event={"ID":"c935dec8-22be-4daf-a60b-fb6087cd1a92","Type":"ContainerStarted","Data":"4c6ced132188911f0a13374b6fe38e3fe560a374c5335ec7d727a7a58ca1d258"} Jul 14 14:37:46.096630 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:46.096578 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" podStartSLOduration=1.096562598 podStartE2EDuration="1.096562598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 14:37:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:37:46.095270612 +0000 UTC m=+2605.321953530" watchObservedRunningTime="2026-07-14 14:37:46.096562598 +0000 UTC m=+2605.323245801" Jul 14 14:37:55.408407 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:55.408374 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:55.408853 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:55.408420 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:55.420813 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:55.420788 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:37:56.141613 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:37:56.141584 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:39:02.208208 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:02.208171 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk"] Jul 14 14:39:17.229607 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.229510 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" podUID="c935dec8-22be-4daf-a60b-fb6087cd1a92" containerName="main" containerID="cri-o://291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea" gracePeriod=45 Jul 14 14:39:17.480145 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.480079 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:39:17.577064 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.577018 2564 generic.go:356] "Generic (PLEG): container finished" podID="c935dec8-22be-4daf-a60b-fb6087cd1a92" containerID="291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea" exitCode=0 Jul 14 14:39:17.577064 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.577066 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" Jul 14 14:39:17.577345 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.577097 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" event={"ID":"c935dec8-22be-4daf-a60b-fb6087cd1a92","Type":"ContainerDied","Data":"291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea"} Jul 14 14:39:17.577345 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.577136 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk" event={"ID":"c935dec8-22be-4daf-a60b-fb6087cd1a92","Type":"ContainerDied","Data":"4c6ced132188911f0a13374b6fe38e3fe560a374c5335ec7d727a7a58ca1d258"} Jul 14 14:39:17.577345 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.577155 2564 scope.go:117] "RemoveContainer" containerID="291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea" Jul 14 14:39:17.586403 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.586383 2564 scope.go:117] "RemoveContainer" containerID="291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea" Jul 14 14:39:17.586727 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:39:17.586705 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea\": container with ID starting with 291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea not found: ID does not exist" containerID="291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea" Jul 14 14:39:17.586807 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.586738 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea"} err="failed to get container status \"291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea\": rpc error: code = NotFound desc = could not find container \"291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea\": container with ID starting with 291c6a6a5970fde444d8f02a9f7b9cc610d62af820e60b3ce9dafbd1e1e143ea not found: ID does not exist" Jul 14 14:39:17.622668 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.622630 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-dshm\") pod \"c935dec8-22be-4daf-a60b-fb6087cd1a92\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " Jul 14 14:39:17.622843 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.622715 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c935dec8-22be-4daf-a60b-fb6087cd1a92-tls-certs\") pod \"c935dec8-22be-4daf-a60b-fb6087cd1a92\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " Jul 14 14:39:17.622843 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.622742 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-home\") pod \"c935dec8-22be-4daf-a60b-fb6087cd1a92\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " Jul 14 14:39:17.622843 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.622780 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbf7z\" (UniqueName: \"kubernetes.io/projected/c935dec8-22be-4daf-a60b-fb6087cd1a92-kube-api-access-dbf7z\") pod \"c935dec8-22be-4daf-a60b-fb6087cd1a92\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " Jul 14 14:39:17.623021 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.622845 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-model-cache\") pod \"c935dec8-22be-4daf-a60b-fb6087cd1a92\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " Jul 14 14:39:17.623021 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.622880 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-tmp-dir\") pod \"c935dec8-22be-4daf-a60b-fb6087cd1a92\" (UID: \"c935dec8-22be-4daf-a60b-fb6087cd1a92\") " Jul 14 14:39:17.623114 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.623034 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-home" (OuterVolumeSpecName: "home") pod "c935dec8-22be-4daf-a60b-fb6087cd1a92" (UID: "c935dec8-22be-4daf-a60b-fb6087cd1a92"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:17.623172 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.623116 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-model-cache" (OuterVolumeSpecName: "model-cache") pod "c935dec8-22be-4daf-a60b-fb6087cd1a92" (UID: "c935dec8-22be-4daf-a60b-fb6087cd1a92"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:17.623238 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.623184 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c935dec8-22be-4daf-a60b-fb6087cd1a92" (UID: "c935dec8-22be-4daf-a60b-fb6087cd1a92"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:17.623358 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.623331 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-tmp-dir\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:17.623430 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.623359 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-home\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:17.623430 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.623372 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-model-cache\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:17.624954 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.624933 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-dshm" (OuterVolumeSpecName: "dshm") pod "c935dec8-22be-4daf-a60b-fb6087cd1a92" (UID: "c935dec8-22be-4daf-a60b-fb6087cd1a92"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:17.624954 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.624944 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c935dec8-22be-4daf-a60b-fb6087cd1a92-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c935dec8-22be-4daf-a60b-fb6087cd1a92" (UID: "c935dec8-22be-4daf-a60b-fb6087cd1a92"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 14:39:17.625119 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.624994 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c935dec8-22be-4daf-a60b-fb6087cd1a92-kube-api-access-dbf7z" (OuterVolumeSpecName: "kube-api-access-dbf7z") pod "c935dec8-22be-4daf-a60b-fb6087cd1a92" (UID: "c935dec8-22be-4daf-a60b-fb6087cd1a92"). InnerVolumeSpecName "kube-api-access-dbf7z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:39:17.723915 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.723880 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c935dec8-22be-4daf-a60b-fb6087cd1a92-dshm\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:17.723915 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.723908 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c935dec8-22be-4daf-a60b-fb6087cd1a92-tls-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:17.723915 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.723919 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbf7z\" (UniqueName: \"kubernetes.io/projected/c935dec8-22be-4daf-a60b-fb6087cd1a92-kube-api-access-dbf7z\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:17.904896 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.904865 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk"] Jul 14 14:39:17.924576 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:17.922406 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8442f2/rate-test-model-8442f2-kserve-7c5549488c-wtfmk"] Jul 14 14:39:19.370651 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:19.370609 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c935dec8-22be-4daf-a60b-fb6087cd1a92" path="/var/lib/kubelet/pods/c935dec8-22be-4daf-a60b-fb6087cd1a92/volumes" Jul 14 14:39:21.516323 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:21.516276 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:39:21.523380 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:21.523358 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kspwb_6954d52c-d756-4559-8c23-5a25abc731f0/ovn-acl-logging/0.log" Jul 14 14:39:48.831327 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:48.831275 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk"] Jul 14 14:39:48.831912 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:48.831507 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" containerID="cri-o://98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818" gracePeriod=30 Jul 14 14:39:54.096881 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.096822 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:39:54.159792 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.159760 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-certs\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.159979 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.159805 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-envoy\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.159979 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.159850 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-data\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.159979 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.159878 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-socket\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.159979 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.159915 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-istio-token\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.159979 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.159976 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-credential-socket\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.160241 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160007 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3666793e-53d0-48cf-b350-f6b3890dda86-istiod-ca-cert\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.160241 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160045 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86gjb\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-kube-api-access-86gjb\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.160241 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160072 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3666793e-53d0-48cf-b350-f6b3890dda86-istio-podinfo\") pod \"3666793e-53d0-48cf-b350-f6b3890dda86\" (UID: \"3666793e-53d0-48cf-b350-f6b3890dda86\") " Jul 14 14:39:54.160241 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160094 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:54.160498 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160242 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:54.160498 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160436 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.160498 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160455 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.160652 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160544 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3666793e-53d0-48cf-b350-f6b3890dda86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:39:54.160652 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160610 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:54.160739 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.160694 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-data" (OuterVolumeSpecName: "istio-data") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:54.162619 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.162592 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3666793e-53d0-48cf-b350-f6b3890dda86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:39:54.162733 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.162641 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:39:54.162733 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.162654 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-istio-token" (OuterVolumeSpecName: "istio-token") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:39:54.162733 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.162670 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-kube-api-access-86gjb" (OuterVolumeSpecName: "kube-api-access-86gjb") pod "3666793e-53d0-48cf-b350-f6b3890dda86" (UID: "3666793e-53d0-48cf-b350-f6b3890dda86"). InnerVolumeSpecName "kube-api-access-86gjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:39:54.261898 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.261859 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.261898 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.261891 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.261898 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.261903 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3666793e-53d0-48cf-b350-f6b3890dda86-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.262154 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.261916 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.262154 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.261928 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3666793e-53d0-48cf-b350-f6b3890dda86-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.262154 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.261939 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86gjb\" (UniqueName: \"kubernetes.io/projected/3666793e-53d0-48cf-b350-f6b3890dda86-kube-api-access-86gjb\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.262154 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.261951 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3666793e-53d0-48cf-b350-f6b3890dda86-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:39:54.788386 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.788342 2564 generic.go:356] "Generic (PLEG): container finished" podID="3666793e-53d0-48cf-b350-f6b3890dda86" containerID="98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818" exitCode=0 Jul 14 14:39:54.788594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.788459 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" Jul 14 14:39:54.788594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.788481 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" event={"ID":"3666793e-53d0-48cf-b350-f6b3890dda86","Type":"ContainerDied","Data":"98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818"} Jul 14 14:39:54.788594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.788538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk" event={"ID":"3666793e-53d0-48cf-b350-f6b3890dda86","Type":"ContainerDied","Data":"ee4d9d798284d3662f968a7dd75014d0ee67792013e86e914a4e0500d0fa5f5f"} Jul 14 14:39:54.788594 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.788561 2564 scope.go:117] "RemoveContainer" containerID="98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818" Jul 14 14:39:54.797793 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.797777 2564 scope.go:117] "RemoveContainer" containerID="8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3" Jul 14 14:39:54.805874 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.805857 2564 scope.go:117] "RemoveContainer" containerID="98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818" Jul 14 14:39:54.806111 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:39:54.806093 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818\": container with ID starting with 98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818 not found: ID does not exist" containerID="98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818" Jul 14 14:39:54.806158 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.806121 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818"} err="failed to get container status \"98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818\": rpc error: code = NotFound desc = could not find container \"98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818\": container with ID starting with 98935d3fc76dd2c19eae3b63491bbbf43f584674c66b59044cdd2e0011ddb818 not found: ID does not exist" Jul 14 14:39:54.806158 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.806139 2564 scope.go:117] "RemoveContainer" containerID="8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3" Jul 14 14:39:54.806410 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:39:54.806389 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3\": container with ID starting with 8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3 not found: ID does not exist" containerID="8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3" Jul 14 14:39:54.806463 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.806416 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3"} err="failed to get container status \"8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3\": rpc error: code = NotFound desc = could not find container \"8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3\": container with ID starting with 8e0fca4d01d16bfd69f42db68c07b7f619e2b8014f737d90ad63dd8c84a754e3 not found: ID does not exist" Jul 14 14:39:54.811861 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.811840 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk"] Jul 14 14:39:54.817666 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:54.817647 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8442f2-openshift-default-646d985b8c-h5zlk"] Jul 14 14:39:55.370238 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:39:55.370201 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" path="/var/lib/kubelet/pods/3666793e-53d0-48cf-b350-f6b3890dda86/volumes" Jul 14 14:40:15.202614 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:15.202576 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4"] Jul 14 14:40:15.203139 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:15.202810 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" containerID="cri-o://2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a" gracePeriod=30 Jul 14 14:40:20.460665 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.460631 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:40:20.620197 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620156 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b848s\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-kube-api-access-b848s\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.620518 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620213 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-socket\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.620518 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620306 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-certs\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.620518 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620339 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-data\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.620518 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620358 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-envoy\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.620518 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620387 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istiod-ca-cert\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.620518 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620410 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-credential-socket\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.620518 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620439 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-token\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.620518 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620467 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-podinfo\") pod \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\" (UID: \"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb\") " Jul 14 14:40:20.621003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620526 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:40:20.621003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620669 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:40:20.621003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620887 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.621003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620907 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-workload-certs\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.621003 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.620963 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 14:40:20.621263 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.621115 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:40:20.621347 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.621301 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:40:20.622665 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.622645 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 14:40:20.622946 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.622923 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-kube-api-access-b848s" (OuterVolumeSpecName: "kube-api-access-b848s") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "kube-api-access-b848s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:40:20.623023 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.622949 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 14:40:20.623066 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.623035 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" (UID: "6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 14:40:20.722098 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.722065 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-data\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.722098 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.722093 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-envoy\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.722098 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.722103 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istiod-ca-cert\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.722347 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.722111 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-credential-socket\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.722347 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.722119 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-token\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.722347 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.722128 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-istio-podinfo\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.722347 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.722135 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b848s\" (UniqueName: \"kubernetes.io/projected/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb-kube-api-access-b848s\") on node \"ip-10-0-142-141.ec2.internal\" DevicePath \"\"" Jul 14 14:40:20.931816 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.931722 2564 generic.go:356] "Generic (PLEG): container finished" podID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerID="2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a" exitCode=0 Jul 14 14:40:20.931816 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.931783 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" Jul 14 14:40:20.931998 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.931806 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" event={"ID":"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb","Type":"ContainerDied","Data":"2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a"} Jul 14 14:40:20.931998 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.931890 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4" event={"ID":"6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb","Type":"ContainerDied","Data":"07a1f775f9e8b4afa01886bcedb54554e255a891f9d57a77e0323f292e8ba33c"} Jul 14 14:40:20.931998 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.931913 2564 scope.go:117] "RemoveContainer" containerID="2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a" Jul 14 14:40:20.941593 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.941577 2564 scope.go:117] "RemoveContainer" containerID="501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62" Jul 14 14:40:20.949805 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.949784 2564 scope.go:117] "RemoveContainer" containerID="2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a" Jul 14 14:40:20.950016 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:40:20.949996 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a\": container with ID starting with 2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a not found: ID does not exist" containerID="2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a" Jul 14 14:40:20.950088 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.950028 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a"} err="failed to get container status \"2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a\": rpc error: code = NotFound desc = could not find container \"2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a\": container with ID starting with 2ff5a801b00f4edb72e421aebc2b70d3e71ee455dd8f99a864058312425c491a not found: ID does not exist" Jul 14 14:40:20.950088 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.950053 2564 scope.go:117] "RemoveContainer" containerID="501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62" Jul 14 14:40:20.950318 ip-10-0-142-141 kubenswrapper[2564]: E0714 14:40:20.950274 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62\": container with ID starting with 501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62 not found: ID does not exist" containerID="501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62" Jul 14 14:40:20.950411 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.950324 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62"} err="failed to get container status \"501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62\": rpc error: code = NotFound desc = could not find container \"501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62\": container with ID starting with 501d475ca1b449defc5969d7378cbce97fe614263143b0b5aa6aa5196ddcfd62 not found: ID does not exist" Jul 14 14:40:20.956582 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.956548 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4"] Jul 14 14:40:20.963863 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:20.963842 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f8f81e-openshift-default-7bfc64bf6c-qbdq4"] Jul 14 14:40:21.369867 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:21.369834 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" path="/var/lib/kubelet/pods/6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb/volumes" Jul 14 14:40:22.141772 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:22.141742 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nnfjj_8464a10c-0d95-42e7-9650-c644e99ed335/server/0.log" Jul 14 14:40:22.551641 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:22.551617 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f858c8968-f9vs5_1b14a204-1245-44b2-87b7-bf6ceb7766d5/copy-manifests/0.log" Jul 14 14:40:22.559021 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:22.558996 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f858c8968-f9vs5_1b14a204-1245-44b2-87b7-bf6ceb7766d5/manager/0.log" Jul 14 14:40:23.260363 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:23.260336 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733975-xk992_31c8d153-2908-4f5a-9e50-3a9d98738d86/cleanup/0.log" Jul 14 14:40:23.387723 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:23.387689 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29733990-rz5x6_9221b79f-5d96-4432-9fdf-7f3febf7ad6c/cleanup/0.log" Jul 14 14:40:23.516170 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:23.516094 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-4f7140c9-g8bgb_5f3add4b-95a5-48ac-97de-483734dfd3ac/revoke-keys/0.log" Jul 14 14:40:24.422364 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:24.422332 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-26a3f6d9-k6qxg_8f3cdffb-e826-4be9-8f1f-0cac719050ca/revoke-keys/0.log" Jul 14 14:40:24.674803 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:24.674723 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-5122206c-dkzmp_f005a1ad-d64b-41af-8fcb-4e339bde6dca/revoke-keys/0.log" Jul 14 14:40:25.950938 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:25.950904 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq_9a56ee5f-38c2-4d6e-86f6-9440960eb188/util/0.log" Jul 14 14:40:25.957549 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:25.957520 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq_9a56ee5f-38c2-4d6e-86f6-9440960eb188/pull/0.log" Jul 14 14:40:25.964783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:25.964757 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq_9a56ee5f-38c2-4d6e-86f6-9440960eb188/extract/0.log" Jul 14 14:40:26.099366 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.099336 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw_3709c140-e4a7-4645-abbf-f5710a658197/util/0.log" Jul 14 14:40:26.107860 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.107833 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw_3709c140-e4a7-4645-abbf-f5710a658197/pull/0.log" Jul 14 14:40:26.115758 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.115738 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw_3709c140-e4a7-4645-abbf-f5710a658197/extract/0.log" Jul 14 14:40:26.250489 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.250411 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk_ecef76ae-1f89-4b0e-8f16-27de8d969c11/extract/0.log" Jul 14 14:40:26.258471 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.258447 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk_ecef76ae-1f89-4b0e-8f16-27de8d969c11/util/0.log" Jul 14 14:40:26.265324 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.265303 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk_ecef76ae-1f89-4b0e-8f16-27de8d969c11/pull/0.log" Jul 14 14:40:26.390528 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.390500 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn_2aac65cf-be49-49c3-b9dc-c61fb45b4fb8/util/0.log" Jul 14 14:40:26.397389 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.397365 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn_2aac65cf-be49-49c3-b9dc-c61fb45b4fb8/pull/0.log" Jul 14 14:40:26.403731 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:26.403715 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn_2aac65cf-be49-49c3-b9dc-c61fb45b4fb8/extract/0.log" Jul 14 14:40:27.926146 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:27.926116 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ld2rn_d7e5c913-2cb3-4baf-a74e-e515c783803f/istio-proxy/0.log" Jul 14 14:40:28.064772 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:28.064739 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pvhmh_3f90acdb-1e88-47af-9e32-b1cb47ae4b62/discovery/0.log" Jul 14 14:40:28.452190 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:28.452157 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4758f_027a7ee3-a371-4302-936c-dd3a282705cf/istio-proxy/1.log" Jul 14 14:40:28.727336 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:28.727223 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-kjrng_809da307-eed7-4f8b-9898-59f57b52cbb7/payload-processing/0.log" Jul 14 14:40:29.232134 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:29.232096 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wrm4s_730b5115-30a9-42c0-8524-df71f30ff430/main/0.log" Jul 14 14:40:29.634647 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:29.634624 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbcwzxk_3bf0d3c9-0272-4953-9671-caa924b0cf28/main/0.log" Jul 14 14:40:37.329752 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:37.329722 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w8zzc_e4f6a09e-45c4-4875-908a-18f27e9c527c/global-pull-secret-syncer/0.log" Jul 14 14:40:37.429012 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:37.428973 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cphc7_68a34d3e-b358-4ff4-88a5-bd3f69f33e55/konnectivity-agent/0.log" Jul 14 14:40:37.535205 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:37.535172 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-141.ec2.internal_e38e3cc78bf216faf085a021c57364a2/haproxy/0.log" Jul 14 14:40:41.112253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.112221 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq_9a56ee5f-38c2-4d6e-86f6-9440960eb188/extract/0.log" Jul 14 14:40:41.137153 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.137129 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq_9a56ee5f-38c2-4d6e-86f6-9440960eb188/util/0.log" Jul 14 14:40:41.160116 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.160090 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759qvjbq_9a56ee5f-38c2-4d6e-86f6-9440960eb188/pull/0.log" Jul 14 14:40:41.278416 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.278390 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw_3709c140-e4a7-4645-abbf-f5710a658197/extract/0.log" Jul 14 14:40:41.301605 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.301580 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw_3709c140-e4a7-4645-abbf-f5710a658197/util/0.log" Jul 14 14:40:41.324639 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.324616 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cf7bw_3709c140-e4a7-4645-abbf-f5710a658197/pull/0.log" Jul 14 14:40:41.352952 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.352929 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk_ecef76ae-1f89-4b0e-8f16-27de8d969c11/extract/0.log" Jul 14 14:40:41.379679 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.379608 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk_ecef76ae-1f89-4b0e-8f16-27de8d969c11/util/0.log" Jul 14 14:40:41.403356 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.403331 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vj2xk_ecef76ae-1f89-4b0e-8f16-27de8d969c11/pull/0.log" Jul 14 14:40:41.433109 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.433077 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn_2aac65cf-be49-49c3-b9dc-c61fb45b4fb8/extract/0.log" Jul 14 14:40:41.454692 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.454668 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn_2aac65cf-be49-49c3-b9dc-c61fb45b4fb8/util/0.log" Jul 14 14:40:41.484848 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:41.484828 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ptmvn_2aac65cf-be49-49c3-b9dc-c61fb45b4fb8/pull/0.log" Jul 14 14:40:44.151604 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.151533 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43215d03-c9bb-45e1-81f3-a0abb88a37e3/alertmanager/0.log" Jul 14 14:40:44.187264 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.187237 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43215d03-c9bb-45e1-81f3-a0abb88a37e3/config-reloader/0.log" Jul 14 14:40:44.220645 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.220616 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43215d03-c9bb-45e1-81f3-a0abb88a37e3/kube-rbac-proxy-web/0.log" Jul 14 14:40:44.245253 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.245229 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43215d03-c9bb-45e1-81f3-a0abb88a37e3/kube-rbac-proxy/0.log" Jul 14 14:40:44.268230 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.268205 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43215d03-c9bb-45e1-81f3-a0abb88a37e3/kube-rbac-proxy-metric/0.log" Jul 14 14:40:44.292539 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.292513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43215d03-c9bb-45e1-81f3-a0abb88a37e3/prom-label-proxy/0.log" Jul 14 14:40:44.316826 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.316802 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43215d03-c9bb-45e1-81f3-a0abb88a37e3/init-config-reloader/0.log" Jul 14 14:40:44.475175 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.475095 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-595cf47db7-fhfld_969f0d24-40f8-409e-9af3-15913ccba965/metrics-server/0.log" Jul 14 14:40:44.502006 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.501979 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-mxhgl_4a1c3119-30ff-422a-bdcc-a576fce3b521/monitoring-plugin/0.log" Jul 14 14:40:44.736492 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.736394 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kwwdb_74f7618b-5c80-40e6-9fee-dc7705cd7dd6/node-exporter/0.log" Jul 14 14:40:44.768250 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.768225 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kwwdb_74f7618b-5c80-40e6-9fee-dc7705cd7dd6/kube-rbac-proxy/0.log" Jul 14 14:40:44.793619 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.793595 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kwwdb_74f7618b-5c80-40e6-9fee-dc7705cd7dd6/init-textfile/0.log" Jul 14 14:40:44.836778 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.836751 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xhf2c_1489a730-84bc-4242-b530-bf640674402b/kube-rbac-proxy-main/0.log" Jul 14 14:40:44.865770 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.865745 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xhf2c_1489a730-84bc-4242-b530-bf640674402b/kube-rbac-proxy-self/0.log" Jul 14 14:40:44.899759 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:44.899739 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-xhf2c_1489a730-84bc-4242-b530-bf640674402b/openshift-state-metrics/0.log" Jul 14 14:40:45.187386 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:45.187361 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vq8th_ae3900f1-13cb-46fa-ab56-20a295609d26/prometheus-operator/0.log" Jul 14 14:40:45.212782 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:45.212757 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-vq8th_ae3900f1-13cb-46fa-ab56-20a295609d26/kube-rbac-proxy/0.log" Jul 14 14:40:45.281783 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:45.281704 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85c45dcbd6-rl5tw_b331fc21-d7c9-45b9-811e-cdd1338cf182/telemeter-client/0.log" Jul 14 14:40:45.307862 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:45.307832 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85c45dcbd6-rl5tw_b331fc21-d7c9-45b9-811e-cdd1338cf182/reload/0.log" Jul 14 14:40:45.332245 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:45.332214 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-85c45dcbd6-rl5tw_b331fc21-d7c9-45b9-811e-cdd1338cf182/kube-rbac-proxy/0.log" Jul 14 14:40:47.633621 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.633586 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8"] Jul 14 14:40:47.634019 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.633997 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" Jul 14 14:40:47.634019 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634015 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" Jul 14 14:40:47.634105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634038 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" Jul 14 14:40:47.634105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634047 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" Jul 14 14:40:47.634105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634069 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" Jul 14 14:40:47.634105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634078 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" Jul 14 14:40:47.634105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634098 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c935dec8-22be-4daf-a60b-fb6087cd1a92" containerName="main" Jul 14 14:40:47.634105 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634103 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c935dec8-22be-4daf-a60b-fb6087cd1a92" containerName="main" Jul 14 14:40:47.634329 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634177 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" Jul 14 14:40:47.634329 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634186 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fd5dfd1-2ebf-4689-b79a-5f706e2a63fb" containerName="istio-proxy" Jul 14 14:40:47.634329 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634198 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3666793e-53d0-48cf-b350-f6b3890dda86" containerName="istio-proxy" Jul 14 14:40:47.634329 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.634209 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c935dec8-22be-4daf-a60b-fb6087cd1a92" containerName="main" Jul 14 14:40:47.637606 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.637591 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.640379 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.640359 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-flfn5\"/\"kube-root-ca.crt\"" Jul 14 14:40:47.640528 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.640464 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-flfn5\"/\"openshift-service-ca.crt\"" Jul 14 14:40:47.641751 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.641732 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-flfn5\"/\"default-dockercfg-6s2zw\"" Jul 14 14:40:47.647051 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.647031 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8"] Jul 14 14:40:47.756542 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.756502 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-proc\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.756728 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.756559 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-podres\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.756728 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.756581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-lib-modules\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.756728 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.756656 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-sys\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.756728 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.756703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzbrx\" (UniqueName: \"kubernetes.io/projected/98664cc5-d437-49e4-85b1-f61158bd8a17-kube-api-access-kzbrx\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.857677 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857640 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzbrx\" (UniqueName: \"kubernetes.io/projected/98664cc5-d437-49e4-85b1-f61158bd8a17-kube-api-access-kzbrx\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.857877 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857700 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-proc\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.857877 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-podres\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.857877 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-lib-modules\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.857877 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857779 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-sys\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.857877 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-proc\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.857877 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857856 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-sys\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.858113 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857915 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-podres\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.858113 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.857918 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98664cc5-d437-49e4-85b1-f61158bd8a17-lib-modules\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.866824 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.866799 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzbrx\" (UniqueName: \"kubernetes.io/projected/98664cc5-d437-49e4-85b1-f61158bd8a17-kube-api-access-kzbrx\") pod \"perf-node-gather-daemonset-zx6z8\" (UID: \"98664cc5-d437-49e4-85b1-f61158bd8a17\") " pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:47.949050 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:47.948950 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:48.085343 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:48.085312 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8"] Jul 14 14:40:48.086704 ip-10-0-142-141 kubenswrapper[2564]: W0714 14:40:48.086672 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod98664cc5_d437_49e4_85b1_f61158bd8a17.slice/crio-c58b42ae5b7fc4f89588c0d6c4848c077d2705996745fa3132c1a42df8722cd3 WatchSource:0}: Error finding container c58b42ae5b7fc4f89588c0d6c4848c077d2705996745fa3132c1a42df8722cd3: Status 404 returned error can't find the container with id c58b42ae5b7fc4f89588c0d6c4848c077d2705996745fa3132c1a42df8722cd3 Jul 14 14:40:49.083388 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:49.083354 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" event={"ID":"98664cc5-d437-49e4-85b1-f61158bd8a17","Type":"ContainerStarted","Data":"1501e5c7aeb0fcde9c81e3a567458d3f16de5b26fb021cd006e8552910962553"} Jul 14 14:40:49.083786 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:49.083409 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" event={"ID":"98664cc5-d437-49e4-85b1-f61158bd8a17","Type":"ContainerStarted","Data":"c58b42ae5b7fc4f89588c0d6c4848c077d2705996745fa3132c1a42df8722cd3"} Jul 14 14:40:49.083786 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:49.083429 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:49.107610 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:49.107559 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" podStartSLOduration=2.107543766 podStartE2EDuration="2.107543766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 14:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 14:40:49.105349231 +0000 UTC m=+2788.332032167" watchObservedRunningTime="2026-07-14 14:40:49.107543766 +0000 UTC m=+2788.334226683" Jul 14 14:40:50.231987 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:50.231959 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-85kbp_487d9f7a-54b6-4d0b-8b43-da0472dc2a3a/dns/0.log" Jul 14 14:40:50.257189 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:50.257158 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-85kbp_487d9f7a-54b6-4d0b-8b43-da0472dc2a3a/kube-rbac-proxy/0.log" Jul 14 14:40:50.438068 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:50.438036 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9nmxt_305e2065-0eb3-424c-b31f-50944b568790/dns-node-resolver/0.log" Jul 14 14:40:51.106856 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:51.106826 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lp6xp_dfe91ad7-93e0-48de-b5b5-58915fce187e/node-ca/0.log" Jul 14 14:40:52.089728 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:52.089700 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ld2rn_d7e5c913-2cb3-4baf-a74e-e515c783803f/istio-proxy/0.log" Jul 14 14:40:52.216416 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:52.216379 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pvhmh_3f90acdb-1e88-47af-9e32-b1cb47ae4b62/discovery/0.log" Jul 14 14:40:52.301814 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:52.301780 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4758f_027a7ee3-a371-4302-936c-dd3a282705cf/istio-proxy/0.log" Jul 14 14:40:52.331013 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:52.330981 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4758f_027a7ee3-a371-4302-936c-dd3a282705cf/istio-proxy/1.log" Jul 14 14:40:52.396422 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:52.396338 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-kjrng_809da307-eed7-4f8b-9898-59f57b52cbb7/payload-processing/0.log" Jul 14 14:40:53.126983 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:53.126950 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22d2x_983e0528-5495-413e-9bcc-73e699b8e85c/serve-healthcheck-canary/0.log" Jul 14 14:40:53.705112 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:53.705074 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdtx6_2b621f22-17e4-4eba-90e6-b84f76091ee9/kube-rbac-proxy/0.log" Jul 14 14:40:53.730475 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:53.730447 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdtx6_2b621f22-17e4-4eba-90e6-b84f76091ee9/exporter/0.log" Jul 14 14:40:53.754044 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:53.754018 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdtx6_2b621f22-17e4-4eba-90e6-b84f76091ee9/extractor/0.log" Jul 14 14:40:55.098745 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:55.098713 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-flfn5/perf-node-gather-daemonset-zx6z8" Jul 14 14:40:56.501887 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:56.501860 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nnfjj_8464a10c-0d95-42e7-9650-c644e99ed335/server/0.log" Jul 14 14:40:56.645579 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:56.645551 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f858c8968-f9vs5_1b14a204-1245-44b2-87b7-bf6ceb7766d5/manager/0.log" Jul 14 14:40:56.667910 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:56.667835 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f858c8968-f9vs5_1b14a204-1245-44b2-87b7-bf6ceb7766d5/copy-manifests/0.log" Jul 14 14:40:58.249030 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:58.249000 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d76b86dd9-nnmjj_5b3fc4be-7361-4cf6-abe5-b8102de94092/manager/0.log" Jul 14 14:40:58.422703 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:58.422668 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ld2rn_d7e5c913-2cb3-4baf-a74e-e515c783803f/istio-proxy/0.log" Jul 14 14:40:58.529472 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:58.529394 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pvhmh_3f90acdb-1e88-47af-9e32-b1cb47ae4b62/discovery/0.log" Jul 14 14:40:58.577393 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:58.577357 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4758f_027a7ee3-a371-4302-936c-dd3a282705cf/istio-proxy/0.log" Jul 14 14:40:58.605879 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:58.605846 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4758f_027a7ee3-a371-4302-936c-dd3a282705cf/istio-proxy/1.log" Jul 14 14:40:58.645933 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:58.645904 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-kjrng_809da307-eed7-4f8b-9898-59f57b52cbb7/payload-processing/0.log" Jul 14 14:40:59.950052 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:40:59.950017 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4ld2rn_d7e5c913-2cb3-4baf-a74e-e515c783803f/istio-proxy/0.log" Jul 14 14:41:00.062733 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:00.062700 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pvhmh_3f90acdb-1e88-47af-9e32-b1cb47ae4b62/discovery/0.log" Jul 14 14:41:00.136946 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:00.136894 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4758f_027a7ee3-a371-4302-936c-dd3a282705cf/istio-proxy/0.log" Jul 14 14:41:00.172624 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:00.170421 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4758f_027a7ee3-a371-4302-936c-dd3a282705cf/istio-proxy/1.log" Jul 14 14:41:00.216670 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:00.216577 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-kjrng_809da307-eed7-4f8b-9898-59f57b52cbb7/payload-processing/0.log" Jul 14 14:41:05.169361 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:05.169330 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nrrq_28a8a35c-90eb-45df-a836-20dd1fdd9ce8/kube-multus-additional-cni-plugins/0.log" Jul 14 14:41:05.201113 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:05.201081 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nrrq_28a8a35c-90eb-45df-a836-20dd1fdd9ce8/egress-router-binary-copy/0.log" Jul 14 14:41:05.228708 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:05.228685 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nrrq_28a8a35c-90eb-45df-a836-20dd1fdd9ce8/cni-plugins/0.log" Jul 14 14:41:05.257088 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:05.257058 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nrrq_28a8a35c-90eb-45df-a836-20dd1fdd9ce8/bond-cni-plugin/0.log" Jul 14 14:41:05.284015 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:05.283985 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nrrq_28a8a35c-90eb-45df-a836-20dd1fdd9ce8/routeoverride-cni/0.log" Jul 14 14:41:05.308854 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:05.308829 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nrrq_28a8a35c-90eb-45df-a836-20dd1fdd9ce8/whereabouts-cni-bincopy/0.log" Jul 14 14:41:05.332225 ip-10-0-142-141 kubenswrapper[2564]: I0714 14:41:05.332197 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7nrrq_28a8a35c-90eb-45df-a836-20dd1fdd9ce8/whereabouts-cni/0.log"