Jul 08 19:21:46.442801 ip-10-0-143-40 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 19:21:46.442814 ip-10-0-143-40 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 19:21:46.442823 ip-10-0-143-40 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 19:21:46.443123 ip-10-0-143-40 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 19:21:56.453073 ip-10-0-143-40 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 19:21:56.453090 ip-10-0-143-40 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 99c7aa5a88414d24a35429629c5e8b42 -- Jul 08 19:23:58.224377 ip-10-0-143-40 systemd[1]: Starting Kubernetes Kubelet... Jul 08 19:24:00.279211 ip-10-0-143-40 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:24:00.279211 ip-10-0-143-40 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 19:24:00.279211 ip-10-0-143-40 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:24:00.279211 ip-10-0-143-40 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 19:24:00.279211 ip-10-0-143-40 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:24:00.280640 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.280129 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 19:24:00.284923 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284895 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:24:00.284923 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284917 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:24:00.284923 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284923 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:24:00.284923 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284931 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284937 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284942 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284947 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284951 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284956 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284959 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284963 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284967 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284971 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284975 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284979 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284983 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284987 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.284991 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285005 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285010 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285014 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285019 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285023 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:24:00.285253 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285027 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285031 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285035 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285040 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285045 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285049 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285053 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285057 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285061 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285065 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285069 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285074 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285078 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285084 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285089 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285109 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285114 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285119 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285123 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285128 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:24:00.286116 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285133 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285137 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285142 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285147 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285151 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285156 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285160 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285164 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285169 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285173 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285177 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285182 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285186 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285191 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285195 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285200 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285204 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285208 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285213 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285217 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:24:00.286998 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285221 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285227 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285231 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285236 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285240 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285245 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285251 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285256 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285262 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285267 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285272 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285276 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285281 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285286 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285290 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285296 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285302 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285314 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285318 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285324 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:24:00.287883 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285329 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285333 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285338 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285343 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285348 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285353 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285358 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285365 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285371 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285376 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285381 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285386 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285393 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285398 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285403 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.285410 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286732 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286745 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286751 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:24:00.288838 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286756 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286764 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286772 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286778 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286784 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286789 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286794 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286798 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286802 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286807 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286812 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286816 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286820 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286825 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286829 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286834 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286838 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286843 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286847 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286852 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:24:00.289733 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286857 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286862 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286867 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286871 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286876 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286882 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286886 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286890 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286895 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286899 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286904 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286909 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286914 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286918 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286923 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286929 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286933 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286938 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286943 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286947 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:24:00.290609 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286952 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286957 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286961 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286966 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286971 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286975 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286980 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286984 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286988 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286993 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.286997 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287001 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287006 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287012 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287017 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287021 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287026 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287030 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287037 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:24:00.291154 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287044 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287049 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287055 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287061 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287067 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287072 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287077 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287082 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287087 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287093 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287117 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287122 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287126 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287131 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287135 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287139 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287144 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287148 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287153 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287159 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:24:00.291725 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287164 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287169 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287173 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287178 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287182 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287191 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287196 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287201 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287207 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287211 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287216 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287220 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287225 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287229 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287236 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287242 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.287247 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288055 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288067 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:24:00.292361 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288072 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288077 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288083 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288088 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288093 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288114 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288120 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288124 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288129 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288133 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288137 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288142 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288146 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288150 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288154 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288160 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288164 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288169 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288174 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288179 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:24:00.293137 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288184 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288189 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288193 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288198 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288203 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288207 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288212 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288216 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288221 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288225 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288233 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288239 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288244 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288249 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288255 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288261 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288268 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288273 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288280 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:24:00.293729 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288285 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288289 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288295 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288300 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288304 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288308 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288313 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288318 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288322 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288327 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288332 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288337 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288342 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288347 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288351 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288356 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288360 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288364 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288369 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288374 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:24:00.294271 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288378 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288383 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288388 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288393 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288397 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288402 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288406 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288410 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288415 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288425 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288430 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288435 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288439 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288446 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288450 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288455 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288459 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288463 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288468 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288473 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:24:00.294911 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288477 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288482 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288487 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288492 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288497 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288501 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288506 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288510 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288515 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288519 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288526 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288532 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288537 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288542 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288547 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288552 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288556 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.288562 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288671 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288683 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 19:24:00.295451 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288699 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288708 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288724 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288730 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288737 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288745 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288757 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288762 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288768 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288774 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288780 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288785 2573 flags.go:64] FLAG: --cgroup-root="" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288790 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288799 2573 flags.go:64] FLAG: --client-ca-file="" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288804 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288810 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288821 2573 flags.go:64] FLAG: --cluster-domain="" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288826 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288832 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288837 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288842 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288849 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288854 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 19:24:00.295957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288859 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288864 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288869 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288876 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288881 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288886 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288891 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288899 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288905 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288910 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288914 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288921 2573 flags.go:64] FLAG: --enable-server="true" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288935 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288957 2573 flags.go:64] FLAG: --event-burst="100" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288963 2573 flags.go:64] FLAG: --event-qps="50" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288968 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288974 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288979 2573 flags.go:64] FLAG: --eviction-hard="" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288992 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.288997 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289002 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289007 2573 flags.go:64] FLAG: --eviction-soft="" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289015 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289020 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289027 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 19:24:00.296557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289032 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289037 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289043 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289047 2573 flags.go:64] FLAG: --feature-gates="" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289055 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289060 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289066 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289071 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289076 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289082 2573 flags.go:64] FLAG: --help="false" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289087 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289092 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289115 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289120 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289126 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289133 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289138 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289143 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289148 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289153 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289173 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289179 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289185 2573 flags.go:64] FLAG: --kube-reserved="" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289190 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 19:24:00.297278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289195 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289201 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289205 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289210 2573 flags.go:64] FLAG: --lock-file="" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289215 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289223 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289229 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289242 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289247 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289252 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289257 2573 flags.go:64] FLAG: --logging-format="text" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289262 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289268 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289278 2573 flags.go:64] FLAG: --manifest-url="" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289284 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289292 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289297 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289305 2573 flags.go:64] FLAG: --max-pods="110" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289310 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289315 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289320 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289326 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289331 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289336 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289341 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 19:24:00.297914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289354 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289360 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289366 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289371 2573 flags.go:64] FLAG: --pod-cidr="" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289385 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289396 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289401 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289406 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289411 2573 flags.go:64] FLAG: --port="10250" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289416 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289421 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06429aaa58c7d49f9" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289427 2573 flags.go:64] FLAG: --qos-reserved="" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289433 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289440 2573 flags.go:64] FLAG: --register-node="true" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289447 2573 flags.go:64] FLAG: --register-with-taints="" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289454 2573 flags.go:64] FLAG: --registry-burst="10" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289459 2573 flags.go:64] FLAG: --registry-qps="5" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289464 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289470 2573 flags.go:64] FLAG: --reserved-memory="" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289476 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289482 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289487 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289493 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289498 2573 flags.go:64] FLAG: --runonce="false" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289503 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 19:24:00.298633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289508 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289514 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289519 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289524 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289530 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289536 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289541 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289546 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289551 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289556 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289562 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289567 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289581 2573 flags.go:64] FLAG: --system-cgroups="" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289586 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289595 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289601 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289605 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289616 2573 flags.go:64] FLAG: --tls-min-version="" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289621 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289626 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289633 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289648 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289655 2573 flags.go:64] FLAG: --v="2" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289663 2573 flags.go:64] FLAG: --version="false" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289670 2573 flags.go:64] FLAG: --vmodule="" Jul 08 19:24:00.299291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289677 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.289682 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289920 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289927 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289932 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289937 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289942 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289947 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289951 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289956 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289961 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289965 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289971 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289975 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289980 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289985 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289990 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289994 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.289999 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:24:00.299953 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290006 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290020 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290028 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290035 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290040 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290046 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290051 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290056 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290064 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290071 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290076 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290081 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290085 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290090 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290114 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290119 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290124 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290128 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290133 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290137 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:24:00.300464 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290143 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290147 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290152 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290156 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290161 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290165 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290170 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290174 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290179 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290184 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290188 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290195 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290202 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290207 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290221 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290227 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290232 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290237 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290241 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:24:00.300988 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290246 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290252 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290260 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290265 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290270 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290274 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290279 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290283 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290287 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290292 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290296 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290301 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290306 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290310 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290315 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290320 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290324 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290329 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290333 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290337 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:24:00.301575 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290342 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290346 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290351 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290355 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290360 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290365 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290370 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290374 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290386 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290391 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290396 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290401 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290405 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290412 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290418 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290423 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290427 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290433 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290437 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290441 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:24:00.302088 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290446 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290451 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.290455 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.291537 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.300035 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.300056 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.300070 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.300076 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300146 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300153 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300156 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300159 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300162 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300165 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:24:00.302650 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300168 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300172 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300174 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300177 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300181 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300183 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300186 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300190 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300192 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300196 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300199 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300202 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300207 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300211 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300214 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300216 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300219 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300223 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300225 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:24:00.303061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300228 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300232 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300235 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300238 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300242 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300245 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300248 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300251 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300254 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300257 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300260 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300263 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300265 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300268 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300272 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300274 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300277 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300280 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300283 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:24:00.303570 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300286 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300289 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300291 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300294 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300297 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300300 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300302 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300305 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300308 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300310 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300313 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300315 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300318 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300321 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300324 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300327 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300330 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300333 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300337 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300340 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:24:00.304230 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300343 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300346 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300349 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300351 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300354 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300357 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300359 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300362 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300365 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300367 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300370 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300374 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300378 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300381 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300384 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300387 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300390 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300393 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300395 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300398 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:24:00.304973 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300400 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300403 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300405 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300409 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300413 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300416 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300420 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300423 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300426 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300429 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300432 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300436 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300438 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300441 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:24:00.305637 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300444 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.300449 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300580 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300586 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300589 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300592 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300595 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300598 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300601 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300604 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300607 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300610 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300613 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300615 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300618 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:24:00.306024 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300621 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300625 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300629 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300632 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300636 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300639 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300643 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300646 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300649 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300652 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300655 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300658 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300660 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300663 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300665 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300669 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300671 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300674 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300677 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300680 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:24:00.306448 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300683 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300685 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300688 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300691 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300694 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300696 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300699 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300702 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300705 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300708 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300711 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300713 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300716 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300719 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300722 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300724 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300727 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300730 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300734 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:24:00.306980 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300736 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300740 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300744 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300747 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300751 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300753 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300756 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300759 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300762 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300765 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300768 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300771 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300774 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300776 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300779 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300782 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300785 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300788 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300790 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300793 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:24:00.307499 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300796 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300799 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300803 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300806 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300809 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300812 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300815 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300818 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300821 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300824 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300827 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300830 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300833 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300835 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300838 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300841 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300844 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300846 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300849 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:24:00.308016 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300851 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300854 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300857 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300860 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300863 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300865 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300868 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:00.300871 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.300876 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.301038 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.303374 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.305510 2573 server.go:1019] "Starting client certificate rotation" Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.305799 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:24:00.308526 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.305848 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:24:00.309177 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.309110 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:24:00.311300 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.311282 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:24:00.324768 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.324702 2573 log.go:25] "Validated CRI v1 runtime API" Jul 08 19:24:00.326995 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.326979 2573 log.go:25] "Validated CRI v1 image API" Jul 08 19:24:00.327839 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.327824 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 19:24:00.330202 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.330173 2573 fs.go:135] Filesystem UUIDs: map[4e2a1cf8-f49e-4e45-98bc-b123329b7b8e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b308505f-7173-4d91-a3bb-0c3ab0fdeab2:/dev/nvme0n1p3] Jul 08 19:24:00.330258 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.330194 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 19:24:00.335342 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.335323 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:24:00.336543 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.336423 2573 manager.go:216] Machine: {Timestamp:2026-07-08 19:24:00.334135009 +0000 UTC m=+1.991573610 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099740 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec273b4cfd64d86f232f9233c53cfcfb SystemUUID:ec273b4c-fd64-d86f-232f-9233c53cfcfb BootID:99c7aa5a-8841-4d24-a354-29629c5e8b42 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2d:43:3a:3c:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2d:43:3a:3c:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:21:0c:42:98:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 19:24:00.336543 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.336538 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 19:24:00.336698 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.336631 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 19:24:00.340022 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.339992 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 19:24:00.340190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.340024 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 19:24:00.340771 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.340760 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 19:24:00.340809 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.340773 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 19:24:00.340809 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.340788 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:24:00.341928 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.341915 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:24:00.343625 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.343613 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 19:24:00.346586 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.346569 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:24:00.346767 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.346756 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 19:24:00.346802 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.346790 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 08 19:24:00.346802 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.346800 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 19:24:00.346872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.346817 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 19:24:00.346872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.346827 2573 kubelet.go:398] "Adding apiserver pod source" Jul 08 19:24:00.346872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.346837 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 19:24:00.347322 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.347289 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:24:00.347322 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.347325 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:24:00.351657 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.351571 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 19:24:00.353493 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.353476 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 19:24:00.353559 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.353511 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 19:24:00.354719 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354705 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 19:24:00.354785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354727 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 19:24:00.354785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354736 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 19:24:00.354785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354744 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 19:24:00.354785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354755 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 19:24:00.354785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354764 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 19:24:00.354785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354775 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 19:24:00.354785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354781 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 19:24:00.354785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354788 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 19:24:00.355126 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354794 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 19:24:00.355126 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354803 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 19:24:00.355126 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.354809 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 19:24:00.355796 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.355785 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 19:24:00.355796 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.355795 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 19:24:00.359350 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.359332 2573 server.go:1268] "Started kubelet" Jul 08 19:24:00.359516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.359454 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 19:24:00.359622 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.359567 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 19:24:00.359686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.359647 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 19:24:00.359686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.359653 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-40.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 19:24:00.359827 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.359798 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 19:24:00.359912 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.359896 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 19:24:00.360434 ip-10-0-143-40 systemd[1]: Started Kubernetes Kubelet. Jul 08 19:24:00.361477 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.361439 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 19:24:00.361820 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.361806 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 08 19:24:00.365583 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.364474 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-40.ec2.internal.18c066f19dcca219 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-40.ec2.internal,UID:ip-10-0-143-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-40.ec2.internal,},FirstTimestamp:2026-07-08 19:24:00.359301657 +0000 UTC m=+2.016740256,LastTimestamp:2026-07-08 19:24:00.359301657 +0000 UTC m=+2.016740256,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-40.ec2.internal,}" Jul 08 19:24:00.366207 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.366186 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 19:24:00.366718 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.366698 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h9bxg" Jul 08 19:24:00.367692 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.367678 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 19:24:00.367749 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.367735 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 19:24:00.367850 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.367824 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:00.368487 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.368325 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 19:24:00.368487 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.368367 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 19:24:00.368487 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.368390 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 19:24:00.368487 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.368486 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 08 19:24:00.368686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.368500 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 08 19:24:00.368779 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.368764 2573 factory.go:55] Registering systemd factory Jul 08 19:24:00.368824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.368784 2573 factory.go:223] Registration of the systemd container factory successfully Jul 08 19:24:00.369034 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.369020 2573 factory.go:153] Registering CRI-O factory Jul 08 19:24:00.369086 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.369039 2573 factory.go:223] Registration of the crio container factory successfully Jul 08 19:24:00.369149 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.369102 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 19:24:00.369149 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.369126 2573 factory.go:103] Registering Raw factory Jul 08 19:24:00.369149 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.369136 2573 manager.go:1196] Started watching for new ooms in manager Jul 08 19:24:00.369884 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.369865 2573 manager.go:349] Starting recovery of all containers Jul 08 19:24:00.376891 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.376862 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h9bxg" Jul 08 19:24:00.380575 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.380536 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 19:24:00.380718 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.380700 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 19:24:00.382461 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.382434 2573 manager.go:354] Recovery completed Jul 08 19:24:00.387170 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.387157 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:24:00.389132 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.389092 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:24:00.389244 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.389149 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:24:00.389244 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.389166 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:24:00.389786 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.389773 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 19:24:00.389786 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.389784 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 19:24:00.389888 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.389787 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:24:00.389888 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.389801 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:24:00.390539 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.390523 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:24:00.390608 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.390551 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:24:00.390608 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.390567 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:24:00.391025 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.391013 2573 policy_none.go:49] "None policy: Start" Jul 08 19:24:00.391057 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.391030 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 19:24:00.391057 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.391041 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 19:24:00.392129 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.392118 2573 policy_none.go:47] "Start" Jul 08 19:24:00.433119 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.432874 2573 manager.go:337] "Starting Device Plugin manager" Jul 08 19:24:00.433276 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.433147 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 19:24:00.433276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.433164 2573 server.go:87] "Starting device plugin registration server" Jul 08 19:24:00.433477 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.433463 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 19:24:00.433520 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.433480 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 19:24:00.433715 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.433689 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 19:24:00.433840 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.433786 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 19:24:00.433840 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.433794 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 19:24:00.434530 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.434509 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 19:24:00.434613 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.434558 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:00.538114 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.538045 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:24:00.540236 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.540218 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:24:00.540341 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.540253 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:24:00.540341 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.540266 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:24:00.540341 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.540294 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.551035 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.550996 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.551035 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.551038 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-40.ec2.internal\": node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:00.559547 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.559515 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 19:24:00.560755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.560734 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 19:24:00.560850 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.560775 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 19:24:00.560850 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.560798 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 19:24:00.560850 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.560837 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 19:24:00.563306 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.563285 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:24:00.567406 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.567382 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:00.661319 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.661206 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal"] Jul 08 19:24:00.661319 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.661323 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:24:00.662394 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.662371 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:24:00.662504 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.662414 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:24:00.662504 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.662432 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:24:00.664260 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.664246 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:24:00.664357 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.664344 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.664400 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.664372 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:24:00.665052 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.665035 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:24:00.665169 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.665067 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:24:00.665169 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.665084 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:24:00.665810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.665791 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:24:00.665893 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.665826 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:24:00.665893 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.665839 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:24:00.668348 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.668331 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:00.669510 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.669493 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.669584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.669519 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:24:00.669640 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.669621 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7532bf8b0b021117e482c83ea0a5feee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal\" (UID: \"7532bf8b0b021117e482c83ea0a5feee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.669691 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.669657 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7532bf8b0b021117e482c83ea0a5feee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal\" (UID: \"7532bf8b0b021117e482c83ea0a5feee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.669743 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.669688 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02f3c755db773470cb39b7721b58aba0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-40.ec2.internal\" (UID: \"02f3c755db773470cb39b7721b58aba0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.670269 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.670250 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:24:00.670375 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.670282 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:24:00.670375 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.670293 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:24:00.703190 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.703163 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-40.ec2.internal\" not found" node="ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.708445 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.708427 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-40.ec2.internal\" not found" node="ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.768944 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.768896 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:00.770156 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.770132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02f3c755db773470cb39b7721b58aba0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-40.ec2.internal\" (UID: \"02f3c755db773470cb39b7721b58aba0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.770219 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.770165 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7532bf8b0b021117e482c83ea0a5feee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal\" (UID: \"7532bf8b0b021117e482c83ea0a5feee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.770219 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.770183 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7532bf8b0b021117e482c83ea0a5feee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal\" (UID: \"7532bf8b0b021117e482c83ea0a5feee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.770299 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.770224 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02f3c755db773470cb39b7721b58aba0-config\") pod \"kube-apiserver-proxy-ip-10-0-143-40.ec2.internal\" (UID: \"02f3c755db773470cb39b7721b58aba0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.770299 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.770224 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7532bf8b0b021117e482c83ea0a5feee-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal\" (UID: \"7532bf8b0b021117e482c83ea0a5feee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.770299 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:00.770290 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7532bf8b0b021117e482c83ea0a5feee-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal\" (UID: \"7532bf8b0b021117e482c83ea0a5feee\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:00.869845 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.869812 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:00.975190 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:00.975120 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.007702 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.007672 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:01.012246 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.011968 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" Jul 08 19:24:01.075912 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.075876 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.176125 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.176094 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.276217 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.276176 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.306661 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.306620 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 19:24:01.307187 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.306802 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 19:24:01.376464 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.376427 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.379732 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.379691 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 19:19:00 +0000 UTC" deadline="2028-03-28 10:01:56.174486868 +0000 UTC" Jul 08 19:24:01.379825 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.379733 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15086h37m54.794757902s" Jul 08 19:24:01.476571 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.476537 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.552915 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:01.552867 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f3c755db773470cb39b7721b58aba0.slice/crio-5266a5d43fcc32106475f3697afb2a843ebf5d9526117c50fd1275231a3d30d4 WatchSource:0}: Error finding container 5266a5d43fcc32106475f3697afb2a843ebf5d9526117c50fd1275231a3d30d4: Status 404 returned error can't find the container with id 5266a5d43fcc32106475f3697afb2a843ebf5d9526117c50fd1275231a3d30d4 Jul 08 19:24:01.557303 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.557283 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:24:01.564147 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.564088 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" event={"ID":"02f3c755db773470cb39b7721b58aba0","Type":"ContainerStarted","Data":"5266a5d43fcc32106475f3697afb2a843ebf5d9526117c50fd1275231a3d30d4"} Jul 08 19:24:01.577034 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.577002 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.677582 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.677555 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.678309 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:01.678282 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7532bf8b0b021117e482c83ea0a5feee.slice/crio-c3dcb522c2ea3db7be9ad82cfa3f544666996f68d5d80c68a776fda9ae49290b WatchSource:0}: Error finding container c3dcb522c2ea3db7be9ad82cfa3f544666996f68d5d80c68a776fda9ae49290b: Status 404 returned error can't find the container with id c3dcb522c2ea3db7be9ad82cfa3f544666996f68d5d80c68a776fda9ae49290b Jul 08 19:24:01.778552 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.778518 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.837019 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.836961 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:24:01.879225 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:01.879179 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-40.ec2.internal\" not found" Jul 08 19:24:01.920749 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.920714 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:24:01.942178 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.942151 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:24:01.968335 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.968298 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" Jul 08 19:24:01.979375 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.979346 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:24:01.980411 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.980389 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" Jul 08 19:24:01.989777 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:01.989758 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:24:02.349187 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.349151 2573 apiserver.go:52] "Watching apiserver" Jul 08 19:24:02.359319 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.359281 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 19:24:02.359759 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.359733 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-nfmvn","openshift-network-operator/iptables-alerter-llplw","openshift-ovn-kubernetes/ovnkube-node-qgfrh","kube-system/konnectivity-agent-nbsqd","kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal","openshift-dns/node-resolver-s9zdv","openshift-image-registry/node-ca-rwqhn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal","openshift-multus/multus-additional-cni-plugins-svsl5","openshift-multus/multus-h2hxc","openshift-multus/network-metrics-daemon-ndsk6","kube-system/global-pull-secret-syncer-ccpl9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p","openshift-cluster-node-tuning-operator/tuned-fzkt8"] Jul 08 19:24:02.361588 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.361561 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.363284 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.363262 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:02.363406 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.363342 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:02.366108 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.364424 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 19:24:02.366108 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.364705 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jmcm6\"" Jul 08 19:24:02.366108 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.364864 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 19:24:02.366108 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.365158 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 19:24:02.366108 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.365506 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.366818 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.367813 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.368393 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.368701 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sblpj\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.368975 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.369169 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.369213 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.369427 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.369457 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.369660 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.369964 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.369994 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.370016 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hjlnm\"" Jul 08 19:24:02.370263 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.370107 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 19:24:02.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.371437 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.372063 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 19:24:02.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.372333 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4rjmq\"" Jul 08 19:24:02.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.372404 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:24:02.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.372428 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ltrq6\"" Jul 08 19:24:02.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.372715 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 19:24:02.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.372751 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 19:24:02.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.372989 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.373769 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.373753 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 19:24:02.373840 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.373769 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 19:24:02.373951 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.373932 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 19:24:02.374135 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.374117 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 19:24:02.374241 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.374221 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7nsz9\"" Jul 08 19:24:02.374594 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.374574 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 19:24:02.375230 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.375206 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 19:24:02.375455 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.375438 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 19:24:02.375549 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.375531 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nngx2\"" Jul 08 19:24:02.375619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.375574 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 19:24:02.376240 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.375816 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.377423 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.377406 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.378503 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.378464 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kctj9\"" Jul 08 19:24:02.378584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.378524 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:24:02.378786 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.378665 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 19:24:02.378786 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.378714 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 19:24:02.379546 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.379524 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 19:24:02.379679 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.379658 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:02.379784 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.379759 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:02.380301 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.380282 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7cf7x\"" Jul 08 19:24:02.381034 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.381009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-slash\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.381158 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.381037 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.381158 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.381050 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-run-netns\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.382153 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.382084 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:02.385505 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385310 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-systemd\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.385505 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-systemd\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.385505 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385421 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr7sz\" (UniqueName: \"kubernetes.io/projected/3c38ec85-addf-4b25-b679-ba6f3c049625-kube-api-access-xr7sz\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.385505 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385453 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:02.385505 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385487 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-ovnkube-script-lib\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385528 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c38ec85-addf-4b25-b679-ba6f3c049625-tmp-dir\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385554 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-node-log\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385585 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phc9t\" (UniqueName: \"kubernetes.io/projected/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-kube-api-access-phc9t\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385618 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-kubernetes\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385649 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6106db48-8fe1-4605-9759-d55a56a5d799-etc-tuned\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c38ec85-addf-4b25-b679-ba6f3c049625-hosts-file\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-serviceca\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385736 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-var-lib-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385764 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-log-socket\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385795 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385834 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l9x7\" (UniqueName: \"kubernetes.io/projected/6106db48-8fe1-4605-9759-d55a56a5d799-kube-api-access-7l9x7\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385860 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-kubelet\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385894 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.385911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385912 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385927 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-sys\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.385962 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-lib-modules\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386086 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-modprobe-d\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386164 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-var-lib-kubelet\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386202 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8w9b\" (UniqueName: \"kubernetes.io/projected/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-kube-api-access-b8w9b\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386237 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-etc-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386290 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysctl-conf\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386319 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-run\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386369 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6106db48-8fe1-4605-9759-d55a56a5d799-tmp\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386419 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-ovn\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-cni-netd\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386502 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-env-overrides\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386545 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d991d98-b53e-49b8-9801-9871673c42cf-ovn-node-metrics-cert\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386586 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/727ac1df-ef0a-459c-ac26-fffd4553bc86-konnectivity-ca\") pod \"konnectivity-agent-nbsqd\" (UID: \"727ac1df-ef0a-459c-ac26-fffd4553bc86\") " pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysctl-d\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386656 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-host\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.386755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386749 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-host\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.387686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386784 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.387686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386832 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-cni-bin\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.387686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386863 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-ovnkube-config\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.387686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386897 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l58pr\" (UniqueName: \"kubernetes.io/projected/3d991d98-b53e-49b8-9801-9871673c42cf-kube-api-access-l58pr\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.387686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.386930 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/727ac1df-ef0a-459c-ac26-fffd4553bc86-agent-certs\") pod \"konnectivity-agent-nbsqd\" (UID: \"727ac1df-ef0a-459c-ac26-fffd4553bc86\") " pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:02.387686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.387001 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysconfig\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.387686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.387038 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-systemd-units\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.470360 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.470319 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 19:24:02.488171 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488121 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-ovnkube-script-lib\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.488338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488177 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpdkw\" (UniqueName: \"kubernetes.io/projected/4a693669-fc51-4915-bbdd-56648c17c496-kube-api-access-kpdkw\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.488338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488212 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-cnibin\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.488338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488246 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-system-cni-dir\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.488338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488274 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt666\" (UniqueName: \"kubernetes.io/projected/5c41a986-2d51-4475-af63-c8f18729ccdf-kube-api-access-wt666\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.488338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.488338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phc9t\" (UniqueName: \"kubernetes.io/projected/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-kube-api-access-phc9t\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488354 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6106db48-8fe1-4605-9759-d55a56a5d799-etc-tuned\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-serviceca\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488403 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-var-lib-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488435 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-log-socket\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-cni-bin\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488491 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-registration-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488515 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-system-cni-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488539 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-socket-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38604f08-ccce-444b-a72f-568e18ce44c9-dbus\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488590 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-lib-modules\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488614 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-modprobe-d\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.488652 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8w9b\" (UniqueName: \"kubernetes.io/projected/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-kube-api-access-b8w9b\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488665 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-cni-netd\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488699 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-netns\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488734 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-multus-certs\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488760 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488784 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-run\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488807 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6106db48-8fe1-4605-9759-d55a56a5d799-tmp\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488830 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-env-overrides\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488855 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-cnibin\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488884 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4a693669-fc51-4915-bbdd-56648c17c496-iptables-alerter-script\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488911 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a693669-fc51-4915-bbdd-56648c17c496-host-slash\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488929 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-host\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488953 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.488990 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-ovnkube-config\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489019 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489047 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-kubelet\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489079 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-systemd-units\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.491225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489118 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-run-netns\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-systemd\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-systemd\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489299 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-host\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489298 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-modprobe-d\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489339 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489829 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-serviceca\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489884 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-var-lib-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489923 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-log-socket\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.489984 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-ovnkube-script-lib\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490044 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-ovnkube-config\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490087 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-lib-modules\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490137 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-os-release\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490146 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-systemd-units\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490169 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a57acc89-7b79-46d7-9483-40eed3dac74f-cni-binary-copy\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490200 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-run-netns\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490416 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-run\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-socket-dir-parent\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.492879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490486 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-cni-multus\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490512 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-hostroot\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490508 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490547 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-systemd\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490596 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-cni-netd\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490606 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-systemd\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490632 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr7sz\" (UniqueName: \"kubernetes.io/projected/3c38ec85-addf-4b25-b679-ba6f3c049625-kube-api-access-xr7sz\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490704 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3d991d98-b53e-49b8-9801-9871673c42cf-env-overrides\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490883 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psp46\" (UniqueName: \"kubernetes.io/projected/8e2c4325-d999-4fe6-a295-568e87f4a5cf-kube-api-access-psp46\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490911 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490938 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c38ec85-addf-4b25-b679-ba6f3c049625-tmp-dir\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490964 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-node-log\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.490999 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-cni-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.491005 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491021 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-k8s-cni-cncf-io\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491043 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-cni-binary-copy\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.493572 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.491120 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:24:02.991063334 +0000 UTC m=+4.648501923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491146 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-node-log\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491156 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491192 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-kubernetes\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c38ec85-addf-4b25-b679-ba6f3c049625-hosts-file\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491234 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c38ec85-addf-4b25-b679-ba6f3c049625-tmp-dir\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491249 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491311 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-kubernetes\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491329 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-daemon-config\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491318 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c38ec85-addf-4b25-b679-ba6f3c049625-hosts-file\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38604f08-ccce-444b-a72f-568e18ce44c9-kubelet-config\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491366 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491426 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9x7\" (UniqueName: \"kubernetes.io/projected/6106db48-8fe1-4605-9759-d55a56a5d799-kube-api-access-7l9x7\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491492 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-kubelet\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491547 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-kubelet\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491586 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491614 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-sys\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.494450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491674 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-sys\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491716 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-var-lib-kubelet\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491771 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-etc-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491799 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-ovn\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491824 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-var-lib-kubelet\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491845 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-os-release\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491873 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-device-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491876 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-etc-openvswitch\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysctl-conf\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491942 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-run-ovn\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d991d98-b53e-49b8-9801-9871673c42cf-ovn-node-metrics-cert\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.491982 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-conf-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492011 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-sys-fs\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492038 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492092 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/727ac1df-ef0a-459c-ac26-fffd4553bc86-konnectivity-ca\") pod \"konnectivity-agent-nbsqd\" (UID: \"727ac1df-ef0a-459c-ac26-fffd4553bc86\") " pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492242 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysctl-conf\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.495325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492289 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysctl-d\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-host\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-cni-bin\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l58pr\" (UniqueName: \"kubernetes.io/projected/3d991d98-b53e-49b8-9801-9871673c42cf-kube-api-access-l58pr\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492401 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdgg8\" (UniqueName: \"kubernetes.io/projected/a57acc89-7b79-46d7-9483-40eed3dac74f-kube-api-access-pdgg8\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492417 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/727ac1df-ef0a-459c-ac26-fffd4553bc86-agent-certs\") pod \"konnectivity-agent-nbsqd\" (UID: \"727ac1df-ef0a-459c-ac26-fffd4553bc86\") " pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492432 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysconfig\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492448 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-slash\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492464 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492482 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-etc-kubernetes\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492590 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/727ac1df-ef0a-459c-ac26-fffd4553bc86-konnectivity-ca\") pod \"konnectivity-agent-nbsqd\" (UID: \"727ac1df-ef0a-459c-ac26-fffd4553bc86\") " pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492637 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysconfig\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492689 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-slash\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492765 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-host\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492772 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3d991d98-b53e-49b8-9801-9871673c42cf-host-cni-bin\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.492780 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6106db48-8fe1-4605-9759-d55a56a5d799-etc-sysctl-d\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.496016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.495954 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/727ac1df-ef0a-459c-ac26-fffd4553bc86-agent-certs\") pod \"konnectivity-agent-nbsqd\" (UID: \"727ac1df-ef0a-459c-ac26-fffd4553bc86\") " pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:02.497133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.497067 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6z9r6" Jul 08 19:24:02.497133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.497090 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6106db48-8fe1-4605-9759-d55a56a5d799-etc-tuned\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.497290 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.497138 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6106db48-8fe1-4605-9759-d55a56a5d799-tmp\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.500227 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.500203 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3d991d98-b53e-49b8-9801-9871673c42cf-ovn-node-metrics-cert\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.504206 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.504179 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr7sz\" (UniqueName: \"kubernetes.io/projected/3c38ec85-addf-4b25-b679-ba6f3c049625-kube-api-access-xr7sz\") pod \"node-resolver-s9zdv\" (UID: \"3c38ec85-addf-4b25-b679-ba6f3c049625\") " pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.505219 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.505192 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6z9r6" Jul 08 19:24:02.506797 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.506769 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phc9t\" (UniqueName: \"kubernetes.io/projected/2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542-kube-api-access-phc9t\") pod \"node-ca-rwqhn\" (UID: \"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542\") " pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.507016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.506983 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8w9b\" (UniqueName: \"kubernetes.io/projected/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-kube-api-access-b8w9b\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:02.507824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.507787 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l58pr\" (UniqueName: \"kubernetes.io/projected/3d991d98-b53e-49b8-9801-9871673c42cf-kube-api-access-l58pr\") pod \"ovnkube-node-qgfrh\" (UID: \"3d991d98-b53e-49b8-9801-9871673c42cf\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.509895 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.509873 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l9x7\" (UniqueName: \"kubernetes.io/projected/6106db48-8fe1-4605-9759-d55a56a5d799-kube-api-access-7l9x7\") pod \"tuned-fzkt8\" (UID: \"6106db48-8fe1-4605-9759-d55a56a5d799\") " pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.568698 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.568653 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" event={"ID":"7532bf8b0b021117e482c83ea0a5feee","Type":"ContainerStarted","Data":"c3dcb522c2ea3db7be9ad82cfa3f544666996f68d5d80c68a776fda9ae49290b"} Jul 08 19:24:02.593338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593299 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdgg8\" (UniqueName: \"kubernetes.io/projected/a57acc89-7b79-46d7-9483-40eed3dac74f-kube-api-access-pdgg8\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593533 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593356 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.593533 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593386 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-etc-kubernetes\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593533 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593413 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpdkw\" (UniqueName: \"kubernetes.io/projected/4a693669-fc51-4915-bbdd-56648c17c496-kube-api-access-kpdkw\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.593533 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593440 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-cnibin\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593533 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593468 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-system-cni-dir\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.593533 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593494 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt666\" (UniqueName: \"kubernetes.io/projected/5c41a986-2d51-4475-af63-c8f18729ccdf-kube-api-access-wt666\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.593533 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593521 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593551 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-cni-bin\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-registration-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-system-cni-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593621 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-cnibin\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-socket-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593657 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38604f08-ccce-444b-a72f-568e18ce44c9-dbus\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593688 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-netns\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593728 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-multus-certs\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-cnibin\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593805 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4a693669-fc51-4915-bbdd-56648c17c496-iptables-alerter-script\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593830 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a693669-fc51-4915-bbdd-56648c17c496-host-slash\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593861 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-kubelet\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.593926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593917 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-os-release\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593942 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a57acc89-7b79-46d7-9483-40eed3dac74f-cni-binary-copy\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593968 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-socket-dir-parent\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.593992 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-cni-multus\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594016 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-hostroot\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594058 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psp46\" (UniqueName: \"kubernetes.io/projected/8e2c4325-d999-4fe6-a295-568e87f4a5cf-kube-api-access-psp46\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594090 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594135 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-cni-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-k8s-cni-cncf-io\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-cni-binary-copy\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594191 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594249 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-daemon-config\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594254 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-etc-kubernetes\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594277 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38604f08-ccce-444b-a72f-568e18ce44c9-kubelet-config\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594326 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-os-release\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594358 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-device-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.594703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594390 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-conf-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594414 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-sys-fs\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594432 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594564 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-socket-dir-parent\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594609 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-os-release\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594724 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594800 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-system-cni-dir\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594806 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-hostroot\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594861 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-etc-selinux\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594928 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-system-cni-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.594938 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-multus-certs\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595006 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.595034 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595051 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c41a986-2d51-4475-af63-c8f18729ccdf-cnibin\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.595081 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret podName:38604f08-ccce-444b-a72f-568e18ce44c9 nodeName:}" failed. No retries permitted until 2026-07-08 19:24:03.095066727 +0000 UTC m=+4.752505312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret") pod "global-pull-secret-syncer-ccpl9" (UID: "38604f08-ccce-444b-a72f-568e18ce44c9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595128 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-registration-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595167 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-daemon-config\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.595575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595186 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a57acc89-7b79-46d7-9483-40eed3dac74f-cni-binary-copy\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595242 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38604f08-ccce-444b-a72f-568e18ce44c9-kubelet-config\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595246 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a693669-fc51-4915-bbdd-56648c17c496-host-slash\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595251 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38604f08-ccce-444b-a72f-568e18ce44c9-dbus\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595261 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-cni-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595308 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-os-release\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595335 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-k8s-cni-cncf-io\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595363 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-device-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595408 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-multus-conf-dir\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595466 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-sys-fs\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595592 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-cni-multus\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595630 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4a693669-fc51-4915-bbdd-56648c17c496-iptables-alerter-script\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595645 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-kubelet\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595691 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-run-netns\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595058 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8e2c4325-d999-4fe6-a295-568e87f4a5cf-socket-dir\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595693 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a57acc89-7b79-46d7-9483-40eed3dac74f-host-var-lib-cni-bin\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595764 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.596374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.595821 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c41a986-2d51-4475-af63-c8f18729ccdf-cni-binary-copy\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.606075 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.606006 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:24:02.606075 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.606034 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:24:02.606075 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.606048 2573 projected.go:196] Error preparing data for projected volume kube-api-access-mnv4g for pod openshift-network-diagnostics/network-check-target-nfmvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:02.606299 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.606140 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g podName:55ff4679-db7b-4d57-bfa3-6c54663761cd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:03.106121596 +0000 UTC m=+4.763560211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mnv4g" (UniqueName: "kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g") pod "network-check-target-nfmvn" (UID: "55ff4679-db7b-4d57-bfa3-6c54663761cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:02.608954 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.608925 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdgg8\" (UniqueName: \"kubernetes.io/projected/a57acc89-7b79-46d7-9483-40eed3dac74f-kube-api-access-pdgg8\") pod \"multus-h2hxc\" (UID: \"a57acc89-7b79-46d7-9483-40eed3dac74f\") " pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.609480 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.609451 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt666\" (UniqueName: \"kubernetes.io/projected/5c41a986-2d51-4475-af63-c8f18729ccdf-kube-api-access-wt666\") pod \"multus-additional-cni-plugins-svsl5\" (UID: \"5c41a986-2d51-4475-af63-c8f18729ccdf\") " pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.611014 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.610989 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psp46\" (UniqueName: \"kubernetes.io/projected/8e2c4325-d999-4fe6-a295-568e87f4a5cf-kube-api-access-psp46\") pod \"aws-ebs-csi-driver-node-znv6p\" (UID: \"8e2c4325-d999-4fe6-a295-568e87f4a5cf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.613227 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.613201 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpdkw\" (UniqueName: \"kubernetes.io/projected/4a693669-fc51-4915-bbdd-56648c17c496-kube-api-access-kpdkw\") pod \"iptables-alerter-llplw\" (UID: \"4a693669-fc51-4915-bbdd-56648c17c496\") " pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.636441 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.636413 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:24:02.675663 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.675628 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rwqhn" Jul 08 19:24:02.687207 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.687163 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:02.697405 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:02.697357 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d991d98_b53e_49b8_9801_9871673c42cf.slice/crio-972a1e08294e1a9c235d5307be2defbca72dbf4df06855cafb083cefda3c9f37 WatchSource:0}: Error finding container 972a1e08294e1a9c235d5307be2defbca72dbf4df06855cafb083cefda3c9f37: Status 404 returned error can't find the container with id 972a1e08294e1a9c235d5307be2defbca72dbf4df06855cafb083cefda3c9f37 Jul 08 19:24:02.698174 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.698091 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:02.705741 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.705554 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" Jul 08 19:24:02.708279 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:02.708255 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod727ac1df_ef0a_459c_ac26_fffd4553bc86.slice/crio-a4b3eee8b5a30322a754cc02d06ddeac952231b267767edf7aa1b978585ed12d WatchSource:0}: Error finding container a4b3eee8b5a30322a754cc02d06ddeac952231b267767edf7aa1b978585ed12d: Status 404 returned error can't find the container with id a4b3eee8b5a30322a754cc02d06ddeac952231b267767edf7aa1b978585ed12d Jul 08 19:24:02.714254 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.714233 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s9zdv" Jul 08 19:24:02.715274 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:02.715247 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6106db48_8fe1_4605_9759_d55a56a5d799.slice/crio-7b70046b803644b37d831537d3a89efa7581c4a8e9dc1a618ae5a3546febf036 WatchSource:0}: Error finding container 7b70046b803644b37d831537d3a89efa7581c4a8e9dc1a618ae5a3546febf036: Status 404 returned error can't find the container with id 7b70046b803644b37d831537d3a89efa7581c4a8e9dc1a618ae5a3546febf036 Jul 08 19:24:02.721886 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.721865 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-svsl5" Jul 08 19:24:02.729926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.729898 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" Jul 08 19:24:02.747745 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.747712 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llplw" Jul 08 19:24:02.754154 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.754125 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h2hxc" Jul 08 19:24:02.997537 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:02.997502 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:02.997729 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.997680 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:02.997784 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:02.997768 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:24:03.997746498 +0000 UTC m=+5.655185108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:03.098666 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.098631 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:03.098831 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:03.098761 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:03.098831 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:03.098822 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret podName:38604f08-ccce-444b-a72f-568e18ce44c9 nodeName:}" failed. No retries permitted until 2026-07-08 19:24:04.098805628 +0000 UTC m=+5.756244235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret") pod "global-pull-secret-syncer-ccpl9" (UID: "38604f08-ccce-444b-a72f-568e18ce44c9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:03.200107 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.199991 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:03.200253 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:03.200180 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:24:03.200253 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:03.200204 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:24:03.200253 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:03.200214 2573 projected.go:196] Error preparing data for projected volume kube-api-access-mnv4g for pod openshift-network-diagnostics/network-check-target-nfmvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:03.200363 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:03.200275 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g podName:55ff4679-db7b-4d57-bfa3-6c54663761cd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:04.200255899 +0000 UTC m=+5.857694507 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnv4g" (UniqueName: "kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g") pod "network-check-target-nfmvn" (UID: "55ff4679-db7b-4d57-bfa3-6c54663761cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:03.509237 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.509187 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:19:02 +0000 UTC" deadline="2028-02-22 17:08:48.873171807 +0000 UTC" Jul 08 19:24:03.509237 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.509229 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14253h44m45.363947555s" Jul 08 19:24:03.561667 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.561627 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:03.561834 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:03.561758 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:03.571495 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.571456 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" event={"ID":"6106db48-8fe1-4605-9759-d55a56a5d799","Type":"ContainerStarted","Data":"7b70046b803644b37d831537d3a89efa7581c4a8e9dc1a618ae5a3546febf036"} Jul 08 19:24:03.572724 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.572701 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nbsqd" event={"ID":"727ac1df-ef0a-459c-ac26-fffd4553bc86","Type":"ContainerStarted","Data":"a4b3eee8b5a30322a754cc02d06ddeac952231b267767edf7aa1b978585ed12d"} Jul 08 19:24:03.573987 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.573950 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"972a1e08294e1a9c235d5307be2defbca72dbf4df06855cafb083cefda3c9f37"} Jul 08 19:24:03.575138 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:03.575111 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rwqhn" event={"ID":"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542","Type":"ContainerStarted","Data":"7084092ca368584136d1833c7cb55731e030db03dda47ee43e284ffbb3f96e6d"} Jul 08 19:24:03.868202 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:03.868170 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c38ec85_addf_4b25_b679_ba6f3c049625.slice/crio-021773ca30c0482b67fa301f25f22e36b990220245424b517b949946b3cd6b04 WatchSource:0}: Error finding container 021773ca30c0482b67fa301f25f22e36b990220245424b517b949946b3cd6b04: Status 404 returned error can't find the container with id 021773ca30c0482b67fa301f25f22e36b990220245424b517b949946b3cd6b04 Jul 08 19:24:03.876560 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:03.876441 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2c4325_d999_4fe6_a295_568e87f4a5cf.slice/crio-5af7c1fa3a298ea585bcb6405b0a4f60f677c9642ddad872e2b7bf1d65c6729e WatchSource:0}: Error finding container 5af7c1fa3a298ea585bcb6405b0a4f60f677c9642ddad872e2b7bf1d65c6729e: Status 404 returned error can't find the container with id 5af7c1fa3a298ea585bcb6405b0a4f60f677c9642ddad872e2b7bf1d65c6729e Jul 08 19:24:03.877145 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:03.877122 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57acc89_7b79_46d7_9483_40eed3dac74f.slice/crio-825d48a86210752725eec21ae94c9dd42fd39b17899348c071f650124f59d497 WatchSource:0}: Error finding container 825d48a86210752725eec21ae94c9dd42fd39b17899348c071f650124f59d497: Status 404 returned error can't find the container with id 825d48a86210752725eec21ae94c9dd42fd39b17899348c071f650124f59d497 Jul 08 19:24:03.878760 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:03.878730 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a693669_fc51_4915_bbdd_56648c17c496.slice/crio-c848afdd9f2fabea9bc4a66b8625e9e1354e9a7bfe6d9513997f8214b79cfa78 WatchSource:0}: Error finding container c848afdd9f2fabea9bc4a66b8625e9e1354e9a7bfe6d9513997f8214b79cfa78: Status 404 returned error can't find the container with id c848afdd9f2fabea9bc4a66b8625e9e1354e9a7bfe6d9513997f8214b79cfa78 Jul 08 19:24:03.879820 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:03.879794 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c41a986_2d51_4475_af63_c8f18729ccdf.slice/crio-6b105f825bc90e798360e42038a635bd9ac53e16e01b22221e9dd49cb75d4cf7 WatchSource:0}: Error finding container 6b105f825bc90e798360e42038a635bd9ac53e16e01b22221e9dd49cb75d4cf7: Status 404 returned error can't find the container with id 6b105f825bc90e798360e42038a635bd9ac53e16e01b22221e9dd49cb75d4cf7 Jul 08 19:24:04.008425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.007565 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:04.008425 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.007723 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:04.008425 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.008033 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:24:06.008009173 +0000 UTC m=+7.665448011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:04.109347 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.109218 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:04.109519 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.109382 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:04.109519 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.109489 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret podName:38604f08-ccce-444b-a72f-568e18ce44c9 nodeName:}" failed. No retries permitted until 2026-07-08 19:24:06.109468587 +0000 UTC m=+7.766907195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret") pod "global-pull-secret-syncer-ccpl9" (UID: "38604f08-ccce-444b-a72f-568e18ce44c9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:04.209997 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.209945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:04.210203 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.210183 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:24:04.210270 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.210205 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:24:04.210270 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.210220 2573 projected.go:196] Error preparing data for projected volume kube-api-access-mnv4g for pod openshift-network-diagnostics/network-check-target-nfmvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:04.210379 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.210295 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g podName:55ff4679-db7b-4d57-bfa3-6c54663761cd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:06.210271625 +0000 UTC m=+7.867710213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnv4g" (UniqueName: "kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g") pod "network-check-target-nfmvn" (UID: "55ff4679-db7b-4d57-bfa3-6c54663761cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:04.509695 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.509624 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:19:02 +0000 UTC" deadline="2028-02-02 17:29:38.541749438 +0000 UTC" Jul 08 19:24:04.509695 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.509693 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13774h5m34.03206072s" Jul 08 19:24:04.562686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.562369 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:04.562686 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.562538 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:04.562686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.562628 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:04.562996 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:04.562755 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:04.591529 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.591478 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" event={"ID":"8e2c4325-d999-4fe6-a295-568e87f4a5cf","Type":"ContainerStarted","Data":"5af7c1fa3a298ea585bcb6405b0a4f60f677c9642ddad872e2b7bf1d65c6729e"} Jul 08 19:24:04.608350 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.608271 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h2hxc" event={"ID":"a57acc89-7b79-46d7-9483-40eed3dac74f","Type":"ContainerStarted","Data":"825d48a86210752725eec21ae94c9dd42fd39b17899348c071f650124f59d497"} Jul 08 19:24:04.619199 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.619154 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s9zdv" event={"ID":"3c38ec85-addf-4b25-b679-ba6f3c049625","Type":"ContainerStarted","Data":"021773ca30c0482b67fa301f25f22e36b990220245424b517b949946b3cd6b04"} Jul 08 19:24:04.636509 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.635605 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" event={"ID":"02f3c755db773470cb39b7721b58aba0","Type":"ContainerStarted","Data":"444ab05219098199788596a79fad4b358fb780fb1fc3ceac13bee1703bdc45a2"} Jul 08 19:24:04.647916 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.647246 2573 generic.go:356] "Generic (PLEG): container finished" podID="7532bf8b0b021117e482c83ea0a5feee" containerID="c0f23359f8e1e3161fb53c343c8fe46e388624a8b8496a68e4516e3b38b88dc3" exitCode=0 Jul 08 19:24:04.647916 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.647343 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" event={"ID":"7532bf8b0b021117e482c83ea0a5feee","Type":"ContainerDied","Data":"c0f23359f8e1e3161fb53c343c8fe46e388624a8b8496a68e4516e3b38b88dc3"} Jul 08 19:24:04.656292 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.655991 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-40.ec2.internal" podStartSLOduration=2.655968133 podStartE2EDuration="2.655968133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:04.655318883 +0000 UTC m=+6.312757493" watchObservedRunningTime="2026-07-08 19:24:04.655968133 +0000 UTC m=+6.313406739" Jul 08 19:24:04.666727 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.666629 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-svsl5" event={"ID":"5c41a986-2d51-4475-af63-c8f18729ccdf","Type":"ContainerStarted","Data":"6b105f825bc90e798360e42038a635bd9ac53e16e01b22221e9dd49cb75d4cf7"} Jul 08 19:24:04.684618 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:04.684519 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llplw" event={"ID":"4a693669-fc51-4915-bbdd-56648c17c496","Type":"ContainerStarted","Data":"c848afdd9f2fabea9bc4a66b8625e9e1354e9a7bfe6d9513997f8214b79cfa78"} Jul 08 19:24:05.562006 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:05.561484 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:05.562006 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:05.561622 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:06.025518 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:06.025476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:06.025732 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.025669 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:06.025794 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.025737 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:24:10.025715143 +0000 UTC m=+11.683153749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:06.126842 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:06.126801 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:06.127078 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.127003 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:06.127143 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.127090 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret podName:38604f08-ccce-444b-a72f-568e18ce44c9 nodeName:}" failed. No retries permitted until 2026-07-08 19:24:10.127054906 +0000 UTC m=+11.784493497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret") pod "global-pull-secret-syncer-ccpl9" (UID: "38604f08-ccce-444b-a72f-568e18ce44c9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:06.227808 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:06.227760 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:06.228553 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.227989 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:24:06.228553 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.228016 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:24:06.228553 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.228029 2573 projected.go:196] Error preparing data for projected volume kube-api-access-mnv4g for pod openshift-network-diagnostics/network-check-target-nfmvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:06.228553 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.228111 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g podName:55ff4679-db7b-4d57-bfa3-6c54663761cd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:10.228091887 +0000 UTC m=+11.885530489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnv4g" (UniqueName: "kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g") pod "network-check-target-nfmvn" (UID: "55ff4679-db7b-4d57-bfa3-6c54663761cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:06.564957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:06.564252 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:06.564957 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.564401 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:06.564957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:06.564793 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:06.564957 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:06.564914 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:07.561847 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:07.561811 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:07.562062 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:07.561969 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:08.561677 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:08.561642 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:08.562203 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:08.561773 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:08.562203 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:08.562111 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:08.562347 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:08.562229 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:09.561690 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:09.561206 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:09.561690 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:09.561343 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:10.060056 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:10.060006 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:10.060229 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.060168 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:10.060286 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.060248 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:24:18.060227881 +0000 UTC m=+19.717666472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:10.160767 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:10.160725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:10.160963 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.160893 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:10.160963 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.160950 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret podName:38604f08-ccce-444b-a72f-568e18ce44c9 nodeName:}" failed. No retries permitted until 2026-07-08 19:24:18.16093254 +0000 UTC m=+19.818371139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret") pod "global-pull-secret-syncer-ccpl9" (UID: "38604f08-ccce-444b-a72f-568e18ce44c9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:10.262449 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:10.261829 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:10.262449 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.261991 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:24:10.262449 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.262012 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:24:10.262449 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.262044 2573 projected.go:196] Error preparing data for projected volume kube-api-access-mnv4g for pod openshift-network-diagnostics/network-check-target-nfmvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:10.262449 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.262107 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g podName:55ff4679-db7b-4d57-bfa3-6c54663761cd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:18.262087838 +0000 UTC m=+19.919526437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnv4g" (UniqueName: "kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g") pod "network-check-target-nfmvn" (UID: "55ff4679-db7b-4d57-bfa3-6c54663761cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:10.562910 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:10.562874 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:10.563382 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:10.562937 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:10.563382 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.563046 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:10.563382 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:10.563143 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:11.561004 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:11.560971 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:11.561204 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:11.561111 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:12.562082 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:12.562001 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:12.562557 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:12.562169 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:12.562557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:12.562214 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:12.562557 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:12.562347 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:13.561541 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:13.561509 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:13.561726 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:13.561642 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:14.562000 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:14.561905 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:14.562467 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:14.562051 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:14.562467 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:14.562055 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:14.562467 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:14.562173 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:15.561545 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:15.561507 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:15.561705 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:15.561632 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:16.561794 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:16.561757 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:16.562281 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:16.561899 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:16.562281 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:16.561947 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:16.562281 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:16.562080 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:17.561241 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:17.561207 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:17.561532 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:17.561320 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:18.123041 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:18.122982 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:18.123503 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.123148 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:18.123503 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.123242 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:24:34.123218118 +0000 UTC m=+35.780656727 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:18.224070 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:18.223979 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:18.224219 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.224149 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:18.224281 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.224224 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret podName:38604f08-ccce-444b-a72f-568e18ce44c9 nodeName:}" failed. No retries permitted until 2026-07-08 19:24:34.224204601 +0000 UTC m=+35.881643200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret") pod "global-pull-secret-syncer-ccpl9" (UID: "38604f08-ccce-444b-a72f-568e18ce44c9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:18.324703 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:18.324671 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:18.324892 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.324863 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:24:18.324892 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.324886 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:24:18.325009 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.324899 2573 projected.go:196] Error preparing data for projected volume kube-api-access-mnv4g for pod openshift-network-diagnostics/network-check-target-nfmvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:18.325009 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.324979 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g podName:55ff4679-db7b-4d57-bfa3-6c54663761cd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:34.32495956 +0000 UTC m=+35.982398157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnv4g" (UniqueName: "kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g") pod "network-check-target-nfmvn" (UID: "55ff4679-db7b-4d57-bfa3-6c54663761cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:18.561962 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:18.561923 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:18.562165 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.562085 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:18.562165 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:18.562155 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:18.562313 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:18.562281 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:19.561824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:19.561784 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:19.562251 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:19.561908 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:20.562347 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:20.562305 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:20.562810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:20.562458 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:20.562810 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:20.562459 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:20.562810 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:20.562551 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:21.561350 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:21.561304 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:21.561644 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:21.561437 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:22.561657 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.561353 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:22.562378 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.561354 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:22.562378 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:22.561760 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:22.562378 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:22.561817 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:22.727215 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.727177 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" event={"ID":"6106db48-8fe1-4605-9759-d55a56a5d799","Type":"ContainerStarted","Data":"a04c81ad721a072c1c0ea182dcc85bbf7ba0ed9160ef00143687c2141c6df351"} Jul 08 19:24:22.728401 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.728377 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nbsqd" event={"ID":"727ac1df-ef0a-459c-ac26-fffd4553bc86","Type":"ContainerStarted","Data":"e51c4fc0e198b5b976c97d5a1c2360c27bb466f2716c2f3d428744192fcaf40a"} Jul 08 19:24:22.729607 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.729588 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rwqhn" event={"ID":"2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542","Type":"ContainerStarted","Data":"d14ff17460fe3f812a4eb8bbd2633af4402c71d57ca9548bfaafbdd0999a76c5"} Jul 08 19:24:22.730938 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.730916 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h2hxc" event={"ID":"a57acc89-7b79-46d7-9483-40eed3dac74f","Type":"ContainerStarted","Data":"4d0609fccf2c522915f180612a60c1bcae092004d8845f48f23c10aec8e04c5e"} Jul 08 19:24:22.732127 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.732107 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s9zdv" event={"ID":"3c38ec85-addf-4b25-b679-ba6f3c049625","Type":"ContainerStarted","Data":"831d8afd5e46d68fb944bb6caed4240e07a431c9c00165f58d4b6e2c120a1869"} Jul 08 19:24:22.733600 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.733579 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" event={"ID":"7532bf8b0b021117e482c83ea0a5feee","Type":"ContainerStarted","Data":"643355af0b5e0fed34c8c78f66bb39f32f59808f484d3d26578efdd9dd9c1bbb"} Jul 08 19:24:22.735578 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.735557 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"c684aa7d20cc5d5884496de1100621ca15bd4cc5b85fbde141d347bdfe6a52ec"} Jul 08 19:24:22.735671 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.735612 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"875b017047c2197c315180876775d2b910d341c792e30dfa17e7985475cd5841"} Jul 08 19:24:22.735671 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.735644 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"3318c66f4f232487591a66225896843184468bcc21af38c135c81d9598892f8b"} Jul 08 19:24:22.735774 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.735677 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"0e6e2fcbea59236faa94725b1f9dc7be9fa6123524456df8066fe574cc91eb6f"} Jul 08 19:24:22.736878 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.736856 2573 generic.go:356] "Generic (PLEG): container finished" podID="5c41a986-2d51-4475-af63-c8f18729ccdf" containerID="37720e4598b1a9f4541b678ab1d8b085be8c5914dc5bd724dccb1c8877301638" exitCode=0 Jul 08 19:24:22.736945 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.736905 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-svsl5" event={"ID":"5c41a986-2d51-4475-af63-c8f18729ccdf","Type":"ContainerDied","Data":"37720e4598b1a9f4541b678ab1d8b085be8c5914dc5bd724dccb1c8877301638"} Jul 08 19:24:22.738433 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.738416 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" event={"ID":"8e2c4325-d999-4fe6-a295-568e87f4a5cf","Type":"ContainerStarted","Data":"12365ba847980d45bfd40c80baa2ddbf0c8b9418bedaae2aa6e0b66258931c80"} Jul 08 19:24:22.742926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.742893 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fzkt8" podStartSLOduration=2.2994044860000002 podStartE2EDuration="21.742860776s" totalImagesPullingTime="19.44345629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:22.742783903 +0000 UTC m=+24.400222530" watchObservedRunningTime="2026-07-08 19:24:22.742860776 +0000 UTC m=+24.400299375" Jul 08 19:24:22.776325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.776289 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s9zdv" podStartSLOduration=3.4879217750000002 podStartE2EDuration="21.776273387s" totalImagesPullingTime="18.288351612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:22.757043733 +0000 UTC m=+24.414482353" watchObservedRunningTime="2026-07-08 19:24:22.776273387 +0000 UTC m=+24.433711971" Jul 08 19:24:22.791729 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.791686 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h2hxc" podStartSLOduration=3.432155926 podStartE2EDuration="21.791671461s" totalImagesPullingTime="18.359515535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:22.791450112 +0000 UTC m=+24.448888735" watchObservedRunningTime="2026-07-08 19:24:22.791671461 +0000 UTC m=+24.449110076" Jul 08 19:24:22.804676 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.804632 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rwqhn" podStartSLOduration=10.543345708 podStartE2EDuration="21.804618008s" totalImagesPullingTime="11.2612723s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:22.804572793 +0000 UTC m=+24.462011400" watchObservedRunningTime="2026-07-08 19:24:22.804618008 +0000 UTC m=+24.462056614" Jul 08 19:24:22.819768 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.819731 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nbsqd" podStartSLOduration=2.372899055 podStartE2EDuration="21.819717422s" totalImagesPullingTime="19.446818367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:22.819424228 +0000 UTC m=+24.476862835" watchObservedRunningTime="2026-07-08 19:24:22.819717422 +0000 UTC m=+24.477156027" Jul 08 19:24:22.839508 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:22.839465 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-40.ec2.internal" podStartSLOduration=20.839449832 podStartE2EDuration="20.839449832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:22.839446096 +0000 UTC m=+24.496884703" watchObservedRunningTime="2026-07-08 19:24:22.839449832 +0000 UTC m=+24.496888439" Jul 08 19:24:23.561227 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:23.561188 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:23.561411 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:23.561311 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:23.743297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:23.743262 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"a8d23719ba6ea2f260ee0c7dbaba0f4aab0d689b211bfcce5d7e5ca40f522151"} Jul 08 19:24:23.743701 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:23.743342 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"cbaeca4c65cf407fece166d140f90ef353b526ba534a56cab2919584e656793e"} Jul 08 19:24:23.744619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:23.744583 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llplw" event={"ID":"4a693669-fc51-4915-bbdd-56648c17c496","Type":"ContainerStarted","Data":"46d9550a9187b86ae3c762a43ec31add709ecb3b316d57cb5ab4149798b8b54e"} Jul 08 19:24:23.760667 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:23.760623 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-llplw" podStartSLOduration=4.4787867519999995 podStartE2EDuration="22.760610685s" totalImagesPullingTime="18.281823933s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:23.760546019 +0000 UTC m=+25.417984627" watchObservedRunningTime="2026-07-08 19:24:23.760610685 +0000 UTC m=+25.418049292" Jul 08 19:24:24.027534 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:24.027507 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 19:24:24.445928 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:24.445811 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T19:24:24.027528881Z","UUID":"8f65e21f-59ee-4fe9-af5a-56d0876e0391","Handler":null,"Name":"","Endpoint":""} Jul 08 19:24:24.447332 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:24.447298 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 19:24:24.447332 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:24.447335 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 19:24:24.561788 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:24.561753 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:24.561978 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:24.561861 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:24.561978 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:24.561880 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:24.561978 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:24.561967 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:24.748540 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:24.748488 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" event={"ID":"8e2c4325-d999-4fe6-a295-568e87f4a5cf","Type":"ContainerStarted","Data":"74a88d19530ebf6a5d3899501b0bf4e8d14b4522d143ae843a196eb37c417fb6"} Jul 08 19:24:25.561629 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:25.561463 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:25.561790 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:25.561742 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:25.657795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:25.657713 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:25.752848 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:25.752814 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"ec9923209792b76d0c08987f357b432f6c6f98945bc476ecdb4e891ec9959d12"} Jul 08 19:24:25.754799 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:25.754777 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" event={"ID":"8e2c4325-d999-4fe6-a295-568e87f4a5cf","Type":"ContainerStarted","Data":"a8ba0a143f05e9e1a4f302a5d25b5b41190bc5ebbe8cf54d2a9ef5ef60a9e4e7"} Jul 08 19:24:25.771975 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:25.771914 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-znv6p" podStartSLOduration=3.700435599 podStartE2EDuration="24.771893657s" totalImagesPullingTime="21.071458058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:25.771367861 +0000 UTC m=+27.428806568" watchObservedRunningTime="2026-07-08 19:24:25.771893657 +0000 UTC m=+27.429332264" Jul 08 19:24:26.561462 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:26.561431 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:26.561607 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:26.561577 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:26.561665 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:26.561633 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:26.562342 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:26.562281 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:26.582425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:26.582392 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:26.583049 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:26.583010 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:26.758079 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:26.758050 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nbsqd" Jul 08 19:24:27.562007 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:27.561825 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:27.562231 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:27.562108 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:27.762961 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:27.762924 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" event={"ID":"3d991d98-b53e-49b8-9801-9871673c42cf","Type":"ContainerStarted","Data":"fcb22bab4a7b93f3b17e6a550fc11363f88fa015ae516f794c6cbddb6e4d27c8"} Jul 08 19:24:27.790397 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:27.790346 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" podStartSLOduration=7.250951065 podStartE2EDuration="26.790329493s" totalImagesPullingTime="19.539378428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:27.788871713 +0000 UTC m=+29.446310332" watchObservedRunningTime="2026-07-08 19:24:27.790329493 +0000 UTC m=+29.447768099" Jul 08 19:24:28.561788 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:28.561759 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:28.561788 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:28.561799 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:28.562042 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:28.561898 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:28.562104 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:28.562044 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:28.765458 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:28.765427 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:28.765906 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:28.765511 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:28.765906 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:28.765551 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:28.781842 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:28.781761 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:28.782086 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:28.781918 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:24:29.237482 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:29.237445 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ccpl9"] Jul 08 19:24:29.237661 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:29.237619 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:29.237753 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:29.237730 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:29.240551 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:29.240512 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nfmvn"] Jul 08 19:24:29.240748 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:29.240633 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:29.240748 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:29.240725 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:29.241317 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:29.241151 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ndsk6"] Jul 08 19:24:29.241317 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:29.241247 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:29.242066 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:29.241347 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:30.562752 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:30.562714 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:30.563190 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:30.562798 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:30.563190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:30.562885 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:30.563190 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:30.563004 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:30.563190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:30.563098 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:30.563325 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:30.563188 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:31.774482 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:31.774297 2573 generic.go:356] "Generic (PLEG): container finished" podID="5c41a986-2d51-4475-af63-c8f18729ccdf" containerID="e5d064231850dcf1bd84ba784668064e0b61f7b27e884b3bdc79521f380600f5" exitCode=0 Jul 08 19:24:31.774943 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:31.774339 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-svsl5" event={"ID":"5c41a986-2d51-4475-af63-c8f18729ccdf","Type":"ContainerDied","Data":"e5d064231850dcf1bd84ba784668064e0b61f7b27e884b3bdc79521f380600f5"} Jul 08 19:24:32.561195 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:32.561161 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:32.561416 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:32.561164 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:32.561416 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:32.561275 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:32.561416 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:32.561169 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:32.561575 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:32.561421 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:32.561575 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:32.561524 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:32.778320 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:32.778284 2573 generic.go:356] "Generic (PLEG): container finished" podID="5c41a986-2d51-4475-af63-c8f18729ccdf" containerID="1bbfc2deb26dbb29dfafdb5b0c54fe6d454a04f17ba9e6cb0bbe565250b3680e" exitCode=0 Jul 08 19:24:32.778844 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:32.778341 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-svsl5" event={"ID":"5c41a986-2d51-4475-af63-c8f18729ccdf","Type":"ContainerDied","Data":"1bbfc2deb26dbb29dfafdb5b0c54fe6d454a04f17ba9e6cb0bbe565250b3680e"} Jul 08 19:24:33.782430 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:33.782396 2573 generic.go:356] "Generic (PLEG): container finished" podID="5c41a986-2d51-4475-af63-c8f18729ccdf" containerID="295a040b6fcd52822845f4df315a1961f85e7f1bb74cfbd68861853d9946cc36" exitCode=0 Jul 08 19:24:33.782942 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:33.782452 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-svsl5" event={"ID":"5c41a986-2d51-4475-af63-c8f18729ccdf","Type":"ContainerDied","Data":"295a040b6fcd52822845f4df315a1961f85e7f1bb74cfbd68861853d9946cc36"} Jul 08 19:24:34.139003 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:34.138916 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:34.139148 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.139062 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:34.139148 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.139114 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:25:06.139101152 +0000 UTC m=+67.796539736 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:24:34.240078 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:34.240040 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:34.240211 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.240158 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:34.240247 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.240241 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret podName:38604f08-ccce-444b-a72f-568e18ce44c9 nodeName:}" failed. No retries permitted until 2026-07-08 19:25:06.24022102 +0000 UTC m=+67.897659610 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret") pod "global-pull-secret-syncer-ccpl9" (UID: "38604f08-ccce-444b-a72f-568e18ce44c9") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:24:34.340620 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:34.340577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:34.340780 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.340746 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:24:34.340780 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.340763 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:24:34.340780 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.340773 2573 projected.go:196] Error preparing data for projected volume kube-api-access-mnv4g for pod openshift-network-diagnostics/network-check-target-nfmvn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:34.340947 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.340830 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g podName:55ff4679-db7b-4d57-bfa3-6c54663761cd nodeName:}" failed. No retries permitted until 2026-07-08 19:25:06.340810535 +0000 UTC m=+67.998249136 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mnv4g" (UniqueName: "kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g") pod "network-check-target-nfmvn" (UID: "55ff4679-db7b-4d57-bfa3-6c54663761cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:24:34.561172 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:34.561136 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:34.561364 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:34.561141 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:34.561364 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.561281 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:24:34.561364 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:34.561141 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:34.561364 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.561346 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ccpl9" podUID="38604f08-ccce-444b-a72f-568e18ce44c9" Jul 08 19:24:34.561564 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:34.561445 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nfmvn" podUID="55ff4679-db7b-4d57-bfa3-6c54663761cd" Jul 08 19:24:35.096461 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.096430 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-40.ec2.internal" event="NodeReady" Jul 08 19:24:35.096927 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.096643 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 19:24:35.146661 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.146631 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gs282"] Jul 08 19:24:35.148523 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.148499 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6x7mh"] Jul 08 19:24:35.148666 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.148636 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.150216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.150198 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6x7mh" Jul 08 19:24:35.153171 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.153149 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dclc4\"" Jul 08 19:24:35.153293 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.153217 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 19:24:35.153651 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.153501 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qgtw2\"" Jul 08 19:24:35.153651 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.153546 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 19:24:35.153819 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.153664 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 19:24:35.153901 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.153874 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 19:24:35.153968 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.153935 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 19:24:35.164140 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.164117 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6x7mh"] Jul 08 19:24:35.165560 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.165496 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gs282"] Jul 08 19:24:35.246883 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.246844 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.247050 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.246901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tg76\" (UniqueName: \"kubernetes.io/projected/bbad0080-35fb-42b3-883d-f81cb83f7789-kube-api-access-5tg76\") pod \"ingress-canary-6x7mh\" (UID: \"bbad0080-35fb-42b3-883d-f81cb83f7789\") " pod="openshift-ingress-canary/ingress-canary-6x7mh" Jul 08 19:24:35.247050 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.246982 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xfcf\" (UniqueName: \"kubernetes.io/projected/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-kube-api-access-5xfcf\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.247140 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.247057 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbad0080-35fb-42b3-883d-f81cb83f7789-cert\") pod \"ingress-canary-6x7mh\" (UID: \"bbad0080-35fb-42b3-883d-f81cb83f7789\") " pod="openshift-ingress-canary/ingress-canary-6x7mh" Jul 08 19:24:35.247140 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.247131 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-config-volume\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.247240 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.247178 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-tmp-dir\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.347684 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.347590 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-tmp-dir\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.347684 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.347645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.347684 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.347677 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tg76\" (UniqueName: \"kubernetes.io/projected/bbad0080-35fb-42b3-883d-f81cb83f7789-kube-api-access-5tg76\") pod \"ingress-canary-6x7mh\" (UID: \"bbad0080-35fb-42b3-883d-f81cb83f7789\") " pod="openshift-ingress-canary/ingress-canary-6x7mh" Jul 08 19:24:35.347964 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.347704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xfcf\" (UniqueName: \"kubernetes.io/projected/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-kube-api-access-5xfcf\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.347964 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.347729 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbad0080-35fb-42b3-883d-f81cb83f7789-cert\") pod \"ingress-canary-6x7mh\" (UID: \"bbad0080-35fb-42b3-883d-f81cb83f7789\") " pod="openshift-ingress-canary/ingress-canary-6x7mh" Jul 08 19:24:35.347964 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.347783 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-config-volume\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.347964 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:35.347793 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:24:35.347964 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:35.347854 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:35.847835622 +0000 UTC m=+37.505274250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:24:35.347964 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.347928 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-tmp-dir\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.348361 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.348343 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-config-volume\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.351793 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.351772 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbad0080-35fb-42b3-883d-f81cb83f7789-cert\") pod \"ingress-canary-6x7mh\" (UID: \"bbad0080-35fb-42b3-883d-f81cb83f7789\") " pod="openshift-ingress-canary/ingress-canary-6x7mh" Jul 08 19:24:35.357008 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.356981 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xfcf\" (UniqueName: \"kubernetes.io/projected/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-kube-api-access-5xfcf\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.357133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.357115 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tg76\" (UniqueName: \"kubernetes.io/projected/bbad0080-35fb-42b3-883d-f81cb83f7789-kube-api-access-5tg76\") pod \"ingress-canary-6x7mh\" (UID: \"bbad0080-35fb-42b3-883d-f81cb83f7789\") " pod="openshift-ingress-canary/ingress-canary-6x7mh" Jul 08 19:24:35.466382 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.466330 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6x7mh" Jul 08 19:24:35.618984 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.618878 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6x7mh"] Jul 08 19:24:35.622935 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:24:35.622906 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbad0080_35fb_42b3_883d_f81cb83f7789.slice/crio-14e5208ef22588b332a51d3869357439b3f188fcda98587458c774a62d3f2a31 WatchSource:0}: Error finding container 14e5208ef22588b332a51d3869357439b3f188fcda98587458c774a62d3f2a31: Status 404 returned error can't find the container with id 14e5208ef22588b332a51d3869357439b3f188fcda98587458c774a62d3f2a31 Jul 08 19:24:35.788462 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.788427 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6x7mh" event={"ID":"bbad0080-35fb-42b3-883d-f81cb83f7789","Type":"ContainerStarted","Data":"14e5208ef22588b332a51d3869357439b3f188fcda98587458c774a62d3f2a31"} Jul 08 19:24:35.851659 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:35.851613 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:35.851859 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:35.851787 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:24:35.851933 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:35.851864 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:36.851843728 +0000 UTC m=+38.509282327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:24:36.561707 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.561673 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:24:36.562383 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.561716 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:24:36.562383 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.561684 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:24:36.565750 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.565727 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:24:36.565900 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.565756 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-98h84\"" Jul 08 19:24:36.566018 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.566004 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:24:36.566427 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.566231 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgx5c\"" Jul 08 19:24:36.566427 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.566280 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:24:36.566427 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.566325 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:24:36.859737 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:36.859643 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:36.859902 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:36.859787 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:24:36.859902 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:36.859851 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:38.859832273 +0000 UTC m=+40.517270871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:24:38.878543 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:38.878497 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:38.878927 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:38.878683 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:24:38.878927 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:38.878747 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:42.878730709 +0000 UTC m=+44.536169298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:24:39.800882 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:39.800683 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6x7mh" event={"ID":"bbad0080-35fb-42b3-883d-f81cb83f7789","Type":"ContainerStarted","Data":"4144ad53dc479d84c16f119f0e51565f4859e7596200a6bfba464bf39a8283f7"} Jul 08 19:24:39.803224 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:39.803199 2573 generic.go:356] "Generic (PLEG): container finished" podID="5c41a986-2d51-4475-af63-c8f18729ccdf" containerID="83bcb6296a9c510a0a6e9b0b8b343c52add72f0c82d72d427a14e7566c77c45e" exitCode=0 Jul 08 19:24:39.803348 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:39.803244 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-svsl5" event={"ID":"5c41a986-2d51-4475-af63-c8f18729ccdf","Type":"ContainerDied","Data":"83bcb6296a9c510a0a6e9b0b8b343c52add72f0c82d72d427a14e7566c77c45e"} Jul 08 19:24:39.816708 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:39.816665 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6x7mh" podStartSLOduration=1.091427416 podStartE2EDuration="4.816651803s" totalImagesPullingTime="3.725224387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:39.816039959 +0000 UTC m=+41.473478650" watchObservedRunningTime="2026-07-08 19:24:39.816651803 +0000 UTC m=+41.474090437" Jul 08 19:24:40.808913 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:40.808880 2573 generic.go:356] "Generic (PLEG): container finished" podID="5c41a986-2d51-4475-af63-c8f18729ccdf" containerID="9e944fac0639c97430a3a9c344ce8e9647d9436046618a8045acba8563fd5214" exitCode=0 Jul 08 19:24:40.809488 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:40.808969 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-svsl5" event={"ID":"5c41a986-2d51-4475-af63-c8f18729ccdf","Type":"ContainerDied","Data":"9e944fac0639c97430a3a9c344ce8e9647d9436046618a8045acba8563fd5214"} Jul 08 19:24:41.814128 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:41.814095 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-svsl5" event={"ID":"5c41a986-2d51-4475-af63-c8f18729ccdf","Type":"ContainerStarted","Data":"d89b5dc3fd4b5be1ae8f04d86d63acee6e4bf2e3143f71b477275bf7768ecda3"} Jul 08 19:24:41.837982 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:41.837936 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-svsl5" podStartSLOduration=7.549057418 podStartE2EDuration="40.837920643s" totalImagesPullingTime="33.288863225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:24:41.836584172 +0000 UTC m=+43.494022797" watchObservedRunningTime="2026-07-08 19:24:41.837920643 +0000 UTC m=+43.495359249" Jul 08 19:24:42.911276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:42.911230 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:42.911736 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:42.911335 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:24:42.911736 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:42.911383 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:24:50.911369628 +0000 UTC m=+52.568808212 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:24:50.964728 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:24:50.964688 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:24:50.965234 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:50.964795 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:24:50.965234 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:24:50.964870 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:25:06.964856839 +0000 UTC m=+68.622295424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:25:00.793010 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:00.792980 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgfrh" Jul 08 19:25:06.175045 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.174987 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:25:06.177725 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.177703 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:25:06.185637 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:25:06.185615 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:25:06.185735 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:25:06.185705 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:26:10.185680831 +0000 UTC m=+131.843119414 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : secret "metrics-daemon-secret" not found Jul 08 19:25:06.275983 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.275940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:25:06.278713 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.278697 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:25:06.290168 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.290139 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38604f08-ccce-444b-a72f-568e18ce44c9-original-pull-secret\") pod \"global-pull-secret-syncer-ccpl9\" (UID: \"38604f08-ccce-444b-a72f-568e18ce44c9\") " pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:25:06.376963 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.376920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:25:06.379379 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.379361 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:25:06.390007 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.389984 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:25:06.401125 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.401098 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnv4g\" (UniqueName: \"kubernetes.io/projected/55ff4679-db7b-4d57-bfa3-6c54663761cd-kube-api-access-mnv4g\") pod \"network-check-target-nfmvn\" (UID: \"55ff4679-db7b-4d57-bfa3-6c54663761cd\") " pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:25:06.586891 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.586859 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ccpl9" Jul 08 19:25:06.595723 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.595700 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-98h84\"" Jul 08 19:25:06.603713 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.603690 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:25:06.721204 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.721163 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ccpl9"] Jul 08 19:25:06.727322 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:25:06.727295 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38604f08_ccce_444b_a72f_568e18ce44c9.slice/crio-d503fc5512fa823ba8e13ec753d44f345aa87565a8d20b120378640491fe776d WatchSource:0}: Error finding container d503fc5512fa823ba8e13ec753d44f345aa87565a8d20b120378640491fe776d: Status 404 returned error can't find the container with id d503fc5512fa823ba8e13ec753d44f345aa87565a8d20b120378640491fe776d Jul 08 19:25:06.734655 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.734628 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nfmvn"] Jul 08 19:25:06.738447 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:25:06.738427 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ff4679_db7b_4d57_bfa3_6c54663761cd.slice/crio-6be0896c43b40141946983f6ed5d89e79d6cbbe0918f07ab4361215c87391d32 WatchSource:0}: Error finding container 6be0896c43b40141946983f6ed5d89e79d6cbbe0918f07ab4361215c87391d32: Status 404 returned error can't find the container with id 6be0896c43b40141946983f6ed5d89e79d6cbbe0918f07ab4361215c87391d32 Jul 08 19:25:06.871779 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.871698 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nfmvn" event={"ID":"55ff4679-db7b-4d57-bfa3-6c54663761cd","Type":"ContainerStarted","Data":"6be0896c43b40141946983f6ed5d89e79d6cbbe0918f07ab4361215c87391d32"} Jul 08 19:25:06.872667 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.872645 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ccpl9" event={"ID":"38604f08-ccce-444b-a72f-568e18ce44c9","Type":"ContainerStarted","Data":"d503fc5512fa823ba8e13ec753d44f345aa87565a8d20b120378640491fe776d"} Jul 08 19:25:06.981044 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:06.980983 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:25:06.981242 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:25:06.981169 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:25:06.981309 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:25:06.981250 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:25:38.981226116 +0000 UTC m=+100.638664711 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:25:10.884663 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:10.884619 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nfmvn" event={"ID":"55ff4679-db7b-4d57-bfa3-6c54663761cd","Type":"ContainerStarted","Data":"87132e28771a64e3c53605fae1c8d927672c6dccfa5ffe672ec23b56edd6ef1f"} Jul 08 19:25:10.885160 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:10.884771 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:25:10.899403 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:10.899344 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nfmvn" podStartSLOduration=66.831619557 podStartE2EDuration="1m9.899325493s" totalImagesPullingTime="3.067705936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:25:10.89905132 +0000 UTC m=+72.556489923" watchObservedRunningTime="2026-07-08 19:25:10.899325493 +0000 UTC m=+72.556764098" Jul 08 19:25:12.890656 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:12.890619 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ccpl9" event={"ID":"38604f08-ccce-444b-a72f-568e18ce44c9","Type":"ContainerStarted","Data":"a9cec4d7c496e7f4175bcad8a1f6b6a953cce6635f441c8ce0146fe8865c92e9"} Jul 08 19:25:12.905553 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:12.905501 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ccpl9" podStartSLOduration=66.365177407 podStartE2EDuration="1m11.905486365s" totalImagesPullingTime="5.540308958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:25:12.904529859 +0000 UTC m=+74.561968481" watchObservedRunningTime="2026-07-08 19:25:12.905486365 +0000 UTC m=+74.562925018" Jul 08 19:25:16.730986 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.730951 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr"] Jul 08 19:25:16.733308 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.733290 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" Jul 08 19:25:16.735748 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.735695 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 19:25:16.735995 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.735960 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-xvhxd\"" Jul 08 19:25:16.736113 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.736066 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 19:25:16.736875 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.736853 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 08 19:25:16.736983 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.736887 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 19:25:16.738148 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.738125 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd"] Jul 08 19:25:16.740454 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.740437 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.742607 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.742589 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 08 19:25:16.742704 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.742687 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 08 19:25:16.742888 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.742873 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 08 19:25:16.742964 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.742875 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 08 19:25:16.745676 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.745658 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr"] Jul 08 19:25:16.752868 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.752848 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd"] Jul 08 19:25:16.854200 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.854154 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.854386 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.854229 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4f4fcc4-88ff-4620-b0c1-12bd322afe13-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c88fff89b-wf6zr\" (UID: \"b4f4fcc4-88ff-4620-b0c1-12bd322afe13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" Jul 08 19:25:16.854386 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.854259 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-ca\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.854386 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.854287 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b4adb4-10ba-4735-8fef-6c5560ff821f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.854386 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.854333 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9h2c\" (UniqueName: \"kubernetes.io/projected/b4f4fcc4-88ff-4620-b0c1-12bd322afe13-kube-api-access-v9h2c\") pod \"managed-serviceaccount-addon-agent-7c88fff89b-wf6zr\" (UID: \"b4f4fcc4-88ff-4620-b0c1-12bd322afe13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" Jul 08 19:25:16.854386 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.854372 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-hub\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.854564 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.854429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm8ww\" (UniqueName: \"kubernetes.io/projected/11b4adb4-10ba-4735-8fef-6c5560ff821f-kube-api-access-tm8ww\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.854564 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.854459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.955459 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.955424 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.955648 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.955464 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.955712 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.955673 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4f4fcc4-88ff-4620-b0c1-12bd322afe13-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c88fff89b-wf6zr\" (UID: \"b4f4fcc4-88ff-4620-b0c1-12bd322afe13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" Jul 08 19:25:16.955760 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.955711 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-ca\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.955760 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.955745 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b4adb4-10ba-4735-8fef-6c5560ff821f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.955905 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.955865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9h2c\" (UniqueName: \"kubernetes.io/projected/b4f4fcc4-88ff-4620-b0c1-12bd322afe13-kube-api-access-v9h2c\") pod \"managed-serviceaccount-addon-agent-7c88fff89b-wf6zr\" (UID: \"b4f4fcc4-88ff-4620-b0c1-12bd322afe13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" Jul 08 19:25:16.956058 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.955910 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-hub\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.956058 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.955986 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ww\" (UniqueName: \"kubernetes.io/projected/11b4adb4-10ba-4735-8fef-6c5560ff821f-kube-api-access-tm8ww\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.956609 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.956584 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b4adb4-10ba-4735-8fef-6c5560ff821f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.958316 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.958286 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.958413 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.958364 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.958413 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.958398 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-ca\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.958413 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.958408 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b4f4fcc4-88ff-4620-b0c1-12bd322afe13-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c88fff89b-wf6zr\" (UID: \"b4f4fcc4-88ff-4620-b0c1-12bd322afe13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" Jul 08 19:25:16.958831 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.958811 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/11b4adb4-10ba-4735-8fef-6c5560ff821f-hub\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.965012 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.964991 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm8ww\" (UniqueName: \"kubernetes.io/projected/11b4adb4-10ba-4735-8fef-6c5560ff821f-kube-api-access-tm8ww\") pod \"cluster-proxy-proxy-agent-656566c579-gw9pd\" (UID: \"11b4adb4-10ba-4735-8fef-6c5560ff821f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:16.965135 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:16.965114 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9h2c\" (UniqueName: \"kubernetes.io/projected/b4f4fcc4-88ff-4620-b0c1-12bd322afe13-kube-api-access-v9h2c\") pod \"managed-serviceaccount-addon-agent-7c88fff89b-wf6zr\" (UID: \"b4f4fcc4-88ff-4620-b0c1-12bd322afe13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" Jul 08 19:25:17.052604 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:17.052564 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" Jul 08 19:25:17.060311 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:17.060281 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:25:17.194907 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:17.194836 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr"] Jul 08 19:25:17.198871 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:25:17.198841 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4f4fcc4_88ff_4620_b0c1_12bd322afe13.slice/crio-e25f2e35f2f72c2012ef678ca8d1c2326cd9e8d988ecccb32d92d6ce9a210cb4 WatchSource:0}: Error finding container e25f2e35f2f72c2012ef678ca8d1c2326cd9e8d988ecccb32d92d6ce9a210cb4: Status 404 returned error can't find the container with id e25f2e35f2f72c2012ef678ca8d1c2326cd9e8d988ecccb32d92d6ce9a210cb4 Jul 08 19:25:17.215017 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:17.214987 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd"] Jul 08 19:25:17.217870 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:25:17.217843 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11b4adb4_10ba_4735_8fef_6c5560ff821f.slice/crio-488cfb47b46f2f556f96861192b0e7fb2b3399ae65c7ad6f1bad7ca21190d0f3 WatchSource:0}: Error finding container 488cfb47b46f2f556f96861192b0e7fb2b3399ae65c7ad6f1bad7ca21190d0f3: Status 404 returned error can't find the container with id 488cfb47b46f2f556f96861192b0e7fb2b3399ae65c7ad6f1bad7ca21190d0f3 Jul 08 19:25:17.903667 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:17.903627 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" event={"ID":"b4f4fcc4-88ff-4620-b0c1-12bd322afe13","Type":"ContainerStarted","Data":"e25f2e35f2f72c2012ef678ca8d1c2326cd9e8d988ecccb32d92d6ce9a210cb4"} Jul 08 19:25:17.904627 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:17.904594 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" event={"ID":"11b4adb4-10ba-4735-8fef-6c5560ff821f","Type":"ContainerStarted","Data":"488cfb47b46f2f556f96861192b0e7fb2b3399ae65c7ad6f1bad7ca21190d0f3"} Jul 08 19:25:20.917069 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:20.917016 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" event={"ID":"b4f4fcc4-88ff-4620-b0c1-12bd322afe13","Type":"ContainerStarted","Data":"a77340219526be81c0675ff8f9228ac3b7b1a83fd947df603d468b2bbd3471bf"} Jul 08 19:25:20.933809 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:20.933690 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" podStartSLOduration=1.99987533 podStartE2EDuration="4.933670451s" totalImagesPullingTime="2.933795121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:25:20.93317565 +0000 UTC m=+82.590614260" watchObservedRunningTime="2026-07-08 19:25:20.933670451 +0000 UTC m=+82.591109058" Jul 08 19:25:21.921909 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:21.921816 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" event={"ID":"11b4adb4-10ba-4735-8fef-6c5560ff821f","Type":"ContainerStarted","Data":"1a8c042ce5f486edbbb747a1e6ad2e4c96121937b818ba7b6079d97d26866ba8"} Jul 08 19:25:21.921909 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:21.921866 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" event={"ID":"11b4adb4-10ba-4735-8fef-6c5560ff821f","Type":"ContainerStarted","Data":"e3a641884ef09aaabe834f1733ce5c67c298102e3d60c05f631e6cdaa59d61eb"} Jul 08 19:25:21.921909 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:21.921882 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" event={"ID":"11b4adb4-10ba-4735-8fef-6c5560ff821f","Type":"ContainerStarted","Data":"09b17c510f3874ee3a4710e3a707b0f91dd43810971b13d9d21f33a68590ff36"} Jul 08 19:25:21.940427 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:21.940369 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" podStartSLOduration=1.6388908039999999 podStartE2EDuration="5.940352908s" totalImagesPullingTime="4.301462104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:25:21.939757233 +0000 UTC m=+83.597195838" watchObservedRunningTime="2026-07-08 19:25:21.940352908 +0000 UTC m=+83.597791514" Jul 08 19:25:39.012150 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:39.012114 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:25:39.012689 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:25:39.012290 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:25:39.012689 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:25:39.012378 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:26:43.012357255 +0000 UTC m=+164.669795843 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:25:41.889666 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:25:41.889635 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nfmvn" Jul 08 19:26:10.226756 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:26:10.226713 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:26:10.227265 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:26:10.226825 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:26:10.227265 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:26:10.226877 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:28:12.226863691 +0000 UTC m=+253.884302275 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : secret "metrics-daemon-secret" not found Jul 08 19:26:38.159617 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:26:38.159571 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gs282" podUID="ab86e6aa-3980-4d63-b431-1b0bfe2caadd" Jul 08 19:26:39.142070 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:26:39.141967 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gs282" Jul 08 19:26:39.576804 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:26:39.576764 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:26:43.051330 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:26:43.051272 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:26:43.051951 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:26:43.051437 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:26:43.051951 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:26:43.051540 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls podName:ab86e6aa-3980-4d63-b431-1b0bfe2caadd nodeName:}" failed. No retries permitted until 2026-07-08 19:28:45.051517904 +0000 UTC m=+286.708956488 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls") pod "dns-default-gs282" (UID: "ab86e6aa-3980-4d63-b431-1b0bfe2caadd") : secret "dns-default-metrics-tls" not found Jul 08 19:26:53.561178 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:26:53.561097 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:27:21.259578 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:27:21.259545 2573 generic.go:356] "Generic (PLEG): container finished" podID="b4f4fcc4-88ff-4620-b0c1-12bd322afe13" containerID="a77340219526be81c0675ff8f9228ac3b7b1a83fd947df603d468b2bbd3471bf" exitCode=255 Jul 08 19:27:21.259998 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:27:21.259616 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" event={"ID":"b4f4fcc4-88ff-4620-b0c1-12bd322afe13","Type":"ContainerDied","Data":"a77340219526be81c0675ff8f9228ac3b7b1a83fd947df603d468b2bbd3471bf"} Jul 08 19:27:21.259998 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:27:21.259903 2573 scope.go:117] "RemoveContainer" containerID="a77340219526be81c0675ff8f9228ac3b7b1a83fd947df603d468b2bbd3471bf" Jul 08 19:27:22.264404 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:27:22.264370 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c88fff89b-wf6zr" event={"ID":"b4f4fcc4-88ff-4620-b0c1-12bd322afe13","Type":"ContainerStarted","Data":"9b9729b963f592d062722333be9e2ad4ee8eb79791d1f6bb9a6d645ffe05e03f"} Jul 08 19:28:12.244811 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:12.244751 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:28:12.245330 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:12.244881 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 19:28:12.245330 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:12.244943 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs podName:582ae813-01ce-4fef-86ea-6dfc6ea19f3d nodeName:}" failed. No retries permitted until 2026-07-08 19:30:14.24492906 +0000 UTC m=+375.902367645 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs") pod "network-metrics-daemon-ndsk6" (UID: "582ae813-01ce-4fef-86ea-6dfc6ea19f3d") : secret "metrics-daemon-secret" not found Jul 08 19:28:18.597691 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.597659 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2stkk"] Jul 08 19:28:18.600791 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.600772 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.603209 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.603182 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 19:28:18.603328 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.603288 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 19:28:18.604174 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.604155 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 19:28:18.604262 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.604156 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5vrqz\"" Jul 08 19:28:18.604262 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.604243 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 19:28:18.612276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.612254 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2stkk"] Jul 08 19:28:18.690993 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.690958 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45c85443-2edc-4833-bc7e-bb7efb5861d7-crio-socket\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.691193 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.691009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42rvx\" (UniqueName: \"kubernetes.io/projected/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-api-access-42rvx\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.691193 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.691101 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45c85443-2edc-4833-bc7e-bb7efb5861d7-data-volume\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.691193 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.691154 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.691308 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.691198 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.792332 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.792301 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.792503 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.792349 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.792503 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.792387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45c85443-2edc-4833-bc7e-bb7efb5861d7-crio-socket\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.792503 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.792410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42rvx\" (UniqueName: \"kubernetes.io/projected/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-api-access-42rvx\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.792503 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.792430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45c85443-2edc-4833-bc7e-bb7efb5861d7-data-volume\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.792503 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:18.792443 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:28:18.792503 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:18.792489 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:28:18.792888 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:18.792527 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:19.292505 +0000 UTC m=+260.949943584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : secret "insights-runtime-extractor-tls" not found Jul 08 19:28:18.792888 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.792493 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/45c85443-2edc-4833-bc7e-bb7efb5861d7-crio-socket\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.792888 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:18.792568 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:19.292548357 +0000 UTC m=+260.949986959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : configmap "kube-rbac-proxy" not found Jul 08 19:28:18.792888 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.792724 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/45c85443-2edc-4833-bc7e-bb7efb5861d7-data-volume\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:18.800501 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:18.800479 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42rvx\" (UniqueName: \"kubernetes.io/projected/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-api-access-42rvx\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:19.295723 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:19.295689 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:19.295888 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:19.295739 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:19.295888 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:19.295827 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:28:19.295888 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:19.295846 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:28:19.295888 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:19.295888 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:20.295873289 +0000 UTC m=+261.953311873 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : configmap "kube-rbac-proxy" not found Jul 08 19:28:19.296019 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:19.295908 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:20.295893935 +0000 UTC m=+261.953332520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : secret "insights-runtime-extractor-tls" not found Jul 08 19:28:20.304782 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:20.304693 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:20.304782 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:20.304742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:20.305233 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:20.304824 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:28:20.305233 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:20.304859 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:28:20.305233 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:20.304876 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:22.304862989 +0000 UTC m=+263.962301573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : configmap "kube-rbac-proxy" not found Jul 08 19:28:20.305233 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:20.304910 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:22.304897424 +0000 UTC m=+263.962336011 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : secret "insights-runtime-extractor-tls" not found Jul 08 19:28:20.774840 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:20.774819 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s9zdv_3c38ec85-addf-4b25-b679-ba6f3c049625/dns-node-resolver/0.log" Jul 08 19:28:21.583952 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:21.583923 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rwqhn_2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542/node-ca/0.log" Jul 08 19:28:22.320344 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:22.320313 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:22.320497 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:22.320361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:22.320497 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:22.320432 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:28:22.320497 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:22.320462 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:28:22.320497 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:22.320491 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:26.320474501 +0000 UTC m=+267.977913102 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : configmap "kube-rbac-proxy" not found Jul 08 19:28:22.320634 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:22.320516 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:26.320502368 +0000 UTC m=+267.977940952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : secret "insights-runtime-extractor-tls" not found Jul 08 19:28:22.376758 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:22.376729 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6x7mh_bbad0080-35fb-42b3-883d-f81cb83f7789/serve-healthcheck-canary/0.log" Jul 08 19:28:26.351952 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:26.351915 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:26.352495 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:26.351964 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:26.352495 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:26.352060 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:28:26.352495 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:26.352095 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:28:26.352495 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:26.352122 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:34.352108607 +0000 UTC m=+276.009547191 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : configmap "kube-rbac-proxy" not found Jul 08 19:28:26.352495 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:26.352146 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:34.352133095 +0000 UTC m=+276.009571679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : secret "insights-runtime-extractor-tls" not found Jul 08 19:28:34.411185 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:34.411132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:34.411668 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:34.411227 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:34.411668 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:34.411310 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:28:34.411668 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:34.411369 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls podName:45c85443-2edc-4833-bc7e-bb7efb5861d7 nodeName:}" failed. No retries permitted until 2026-07-08 19:28:50.411355923 +0000 UTC m=+292.068794508 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2stkk" (UID: "45c85443-2edc-4833-bc7e-bb7efb5861d7") : secret "insights-runtime-extractor-tls" not found Jul 08 19:28:34.411668 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:34.411606 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/45c85443-2edc-4833-bc7e-bb7efb5861d7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:42.142844 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:42.142780 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gs282" podUID="ab86e6aa-3980-4d63-b431-1b0bfe2caadd" Jul 08 19:28:42.488812 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:42.488727 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gs282" Jul 08 19:28:45.088281 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:45.088242 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:28:45.090499 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:45.090480 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab86e6aa-3980-4d63-b431-1b0bfe2caadd-metrics-tls\") pod \"dns-default-gs282\" (UID: \"ab86e6aa-3980-4d63-b431-1b0bfe2caadd\") " pod="openshift-dns/dns-default-gs282" Jul 08 19:28:45.192681 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:45.192651 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dclc4\"" Jul 08 19:28:45.200835 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:45.200818 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gs282" Jul 08 19:28:45.326931 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:45.326896 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gs282"] Jul 08 19:28:45.331620 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:28:45.331588 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab86e6aa_3980_4d63_b431_1b0bfe2caadd.slice/crio-5df84af0126dff3df843fedb4e6a63869c8cb96803d674edbbd317ca3b580ce8 WatchSource:0}: Error finding container 5df84af0126dff3df843fedb4e6a63869c8cb96803d674edbbd317ca3b580ce8: Status 404 returned error can't find the container with id 5df84af0126dff3df843fedb4e6a63869c8cb96803d674edbbd317ca3b580ce8 Jul 08 19:28:45.497876 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:45.497840 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gs282" event={"ID":"ab86e6aa-3980-4d63-b431-1b0bfe2caadd","Type":"ContainerStarted","Data":"5df84af0126dff3df843fedb4e6a63869c8cb96803d674edbbd317ca3b580ce8"} Jul 08 19:28:47.505761 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:47.505715 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gs282" event={"ID":"ab86e6aa-3980-4d63-b431-1b0bfe2caadd","Type":"ContainerStarted","Data":"fe64cf3e764b6018703e3e8d4da33199f3b66d2c0d9a440e6039f000ccb8b5a8"} Jul 08 19:28:47.506193 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:47.505817 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gs282" event={"ID":"ab86e6aa-3980-4d63-b431-1b0bfe2caadd","Type":"ContainerStarted","Data":"137f6f66d51b59d635cbeb1a5b1afda2c9a6bde167409259c5d7e063c781fe17"} Jul 08 19:28:47.506193 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:47.505848 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gs282" Jul 08 19:28:47.540434 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:47.540386 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gs282" podStartSLOduration=250.922274775 podStartE2EDuration="4m12.540371867s" totalImagesPullingTime="1.618097092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:28:47.538513172 +0000 UTC m=+289.195951778" watchObservedRunningTime="2026-07-08 19:28:47.540371867 +0000 UTC m=+289.197810473" Jul 08 19:28:50.431868 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:50.431822 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:50.434168 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:50.434145 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/45c85443-2edc-4833-bc7e-bb7efb5861d7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2stkk\" (UID: \"45c85443-2edc-4833-bc7e-bb7efb5861d7\") " pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:50.709485 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:50.709399 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2stkk" Jul 08 19:28:50.836305 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:50.836274 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2stkk"] Jul 08 19:28:50.839537 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:28:50.839511 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c85443_2edc_4833_bc7e_bb7efb5861d7.slice/crio-6bcc9c16d96f852c99c65a1d73a038a4f4bb811f36743d967dbe8c7aa06473e8 WatchSource:0}: Error finding container 6bcc9c16d96f852c99c65a1d73a038a4f4bb811f36743d967dbe8c7aa06473e8: Status 404 returned error can't find the container with id 6bcc9c16d96f852c99c65a1d73a038a4f4bb811f36743d967dbe8c7aa06473e8 Jul 08 19:28:51.524041 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:51.523994 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2stkk" event={"ID":"45c85443-2edc-4833-bc7e-bb7efb5861d7","Type":"ContainerStarted","Data":"6bcc9c16d96f852c99c65a1d73a038a4f4bb811f36743d967dbe8c7aa06473e8"} Jul 08 19:28:55.537982 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:55.537947 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2stkk" event={"ID":"45c85443-2edc-4833-bc7e-bb7efb5861d7","Type":"ContainerStarted","Data":"9e2b17864d97748d37af08cae4990be16f41f4fa465db7d47c9d449922abc775"} Jul 08 19:28:55.538378 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:55.538001 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2stkk" event={"ID":"45c85443-2edc-4833-bc7e-bb7efb5861d7","Type":"ContainerStarted","Data":"dc5f4e8608584a9b50b26167941ba3adb2efb8660f84a7f4eb74fc0f6ee9b172"} Jul 08 19:28:56.562479 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:28:56.562433 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ndsk6" podUID="582ae813-01ce-4fef-86ea-6dfc6ea19f3d" Jul 08 19:28:57.511109 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:57.511080 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gs282" Jul 08 19:28:58.550395 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:58.550352 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2stkk" event={"ID":"45c85443-2edc-4833-bc7e-bb7efb5861d7","Type":"ContainerStarted","Data":"53ad7ca0bdbcc840ec4413cba402b09fafed91dfdd5e3d61d35d90c8342dd14f"} Jul 08 19:28:58.570793 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:58.570743 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2stkk" podStartSLOduration=33.803091862 podStartE2EDuration="40.570728861s" totalImagesPullingTime="6.767636999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:28:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:28:58.569221189 +0000 UTC m=+300.226659812" watchObservedRunningTime="2026-07-08 19:28:58.570728861 +0000 UTC m=+300.228167466" Jul 08 19:28:59.317079 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.317040 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mtpkd"] Jul 08 19:28:59.320202 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.320182 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.322849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.322825 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 19:28:59.322986 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.322966 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 19:28:59.323202 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.323178 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 19:28:59.324005 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.323969 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 19:28:59.324005 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.323992 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 19:28:59.324177 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.324060 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 19:28:59.324177 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.324081 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7zzsr\"" Jul 08 19:28:59.400658 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400625 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-metrics-client-ca\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.400872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.400872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400713 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-tls\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.400872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400750 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.400872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400796 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbbtq\" (UniqueName: \"kubernetes.io/projected/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-kube-api-access-qbbtq\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.400872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400845 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-textfile\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.400872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400864 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-wtmp\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.401129 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400881 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-sys\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.401129 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.400905 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-root\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502298 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-metrics-client-ca\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502480 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502314 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502480 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502346 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-tls\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502480 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502480 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502404 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbbtq\" (UniqueName: \"kubernetes.io/projected/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-kube-api-access-qbbtq\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502687 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502652 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-textfile\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502749 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-wtmp\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502749 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-sys\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502847 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-root\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502896 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502876 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-root\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502899 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-wtmp\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.502957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502919 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-sys\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.503085 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502970 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-textfile\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.503085 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.502996 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-metrics-client-ca\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.503085 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.503043 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-accelerators-collector-config\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.504807 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.504785 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-tls\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.504928 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.504827 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.511734 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.511713 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbbtq\" (UniqueName: \"kubernetes.io/projected/3615a4c9-ae12-409e-ac2d-bb3a94eba7d0-kube-api-access-qbbtq\") pod \"node-exporter-mtpkd\" (UID: \"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0\") " pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.629206 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:28:59.629112 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mtpkd" Jul 08 19:28:59.640782 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:28:59.640758 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3615a4c9_ae12_409e_ac2d_bb3a94eba7d0.slice/crio-33762b43ac5d0e6df4f3ce22423409fa68b4a44bc4eaea09d39b4f7884a9a002 WatchSource:0}: Error finding container 33762b43ac5d0e6df4f3ce22423409fa68b4a44bc4eaea09d39b4f7884a9a002: Status 404 returned error can't find the container with id 33762b43ac5d0e6df4f3ce22423409fa68b4a44bc4eaea09d39b4f7884a9a002 Jul 08 19:29:00.391644 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:00.391616 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 19:29:00.558073 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:00.557905 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mtpkd" event={"ID":"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0","Type":"ContainerStarted","Data":"33762b43ac5d0e6df4f3ce22423409fa68b4a44bc4eaea09d39b4f7884a9a002"} Jul 08 19:29:01.418457 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.418421 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-8p4w4"] Jul 08 19:29:01.421237 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.421221 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-8p4w4" Jul 08 19:29:01.423543 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.423522 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 19:29:01.423543 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.423528 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-x2kzz\"" Jul 08 19:29:01.423705 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.423629 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 19:29:01.431651 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.431626 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-8p4w4"] Jul 08 19:29:01.520489 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.520445 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x9gq\" (UniqueName: \"kubernetes.io/projected/d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a-kube-api-access-4x9gq\") pod \"downloads-685bfb5f64-8p4w4\" (UID: \"d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a\") " pod="openshift-console/downloads-685bfb5f64-8p4w4" Jul 08 19:29:01.562222 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.562191 2573 generic.go:356] "Generic (PLEG): container finished" podID="3615a4c9-ae12-409e-ac2d-bb3a94eba7d0" containerID="87873d067d06834b3bd4444bdcb54db8ae2da0b2bdcb076335dc74b21c317e88" exitCode=0 Jul 08 19:29:01.562375 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.562279 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mtpkd" event={"ID":"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0","Type":"ContainerDied","Data":"87873d067d06834b3bd4444bdcb54db8ae2da0b2bdcb076335dc74b21c317e88"} Jul 08 19:29:01.620861 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.620836 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x9gq\" (UniqueName: \"kubernetes.io/projected/d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a-kube-api-access-4x9gq\") pod \"downloads-685bfb5f64-8p4w4\" (UID: \"d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a\") " pod="openshift-console/downloads-685bfb5f64-8p4w4" Jul 08 19:29:01.631957 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.631926 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x9gq\" (UniqueName: \"kubernetes.io/projected/d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a-kube-api-access-4x9gq\") pod \"downloads-685bfb5f64-8p4w4\" (UID: \"d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a\") " pod="openshift-console/downloads-685bfb5f64-8p4w4" Jul 08 19:29:01.729576 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.729533 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-8p4w4" Jul 08 19:29:01.858733 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.858680 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-8p4w4"] Jul 08 19:29:01.861795 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:29:01.861767 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c477c4_df46_4dc0_9a1c_dceeeb0fb04a.slice/crio-8f75552a4bf23f975376e754b6e11823793e6fb1ddeb582467c60ee986604d2b WatchSource:0}: Error finding container 8f75552a4bf23f975376e754b6e11823793e6fb1ddeb582467c60ee986604d2b: Status 404 returned error can't find the container with id 8f75552a4bf23f975376e754b6e11823793e6fb1ddeb582467c60ee986604d2b Jul 08 19:29:01.863731 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:01.863716 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:29:02.567761 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:02.567723 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-8p4w4" event={"ID":"d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a","Type":"ContainerStarted","Data":"8f75552a4bf23f975376e754b6e11823793e6fb1ddeb582467c60ee986604d2b"} Jul 08 19:29:02.569704 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:02.569679 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mtpkd" event={"ID":"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0","Type":"ContainerStarted","Data":"65b4dc2137b34b6200c87c07dd5135e48b547ce5a78d24ffeaa466a0ee4d1773"} Jul 08 19:29:02.569821 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:02.569726 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mtpkd" event={"ID":"3615a4c9-ae12-409e-ac2d-bb3a94eba7d0","Type":"ContainerStarted","Data":"ca1f32c3968b96e2ab97ac8a7878104085b2b40f8d4874f16decfa3d64d2376c"} Jul 08 19:29:02.595359 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:02.595297 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mtpkd" podStartSLOduration=2.477091457 podStartE2EDuration="3.595276474s" totalImagesPullingTime="1.052734655s" totalInitContainerRuntime="65.450362ms" isStatefulPod=true podCreationTimestamp="2026-07-08 19:28:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:29:02.594413838 +0000 UTC m=+304.251852499" watchObservedRunningTime="2026-07-08 19:29:02.595276474 +0000 UTC m=+304.252715082" Jul 08 19:29:04.014577 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.014538 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n"] Jul 08 19:29:04.017385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.017367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:04.020110 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.020087 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rqwzr\"" Jul 08 19:29:04.020110 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.020107 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 19:29:04.033069 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.033044 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n"] Jul 08 19:29:04.142594 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.142544 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwvqs\" (UniqueName: \"kubernetes.io/projected/c947590c-8931-41a1-b313-183c2d23d223-kube-api-access-bwvqs\") pod \"monitoring-plugin-d5dcff7cf-knm4n\" (UID: \"c947590c-8931-41a1-b313-183c2d23d223\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:04.142776 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.142617 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c947590c-8931-41a1-b313-183c2d23d223-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-knm4n\" (UID: \"c947590c-8931-41a1-b313-183c2d23d223\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:04.243866 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.243440 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c947590c-8931-41a1-b313-183c2d23d223-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-knm4n\" (UID: \"c947590c-8931-41a1-b313-183c2d23d223\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:04.243866 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.243545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwvqs\" (UniqueName: \"kubernetes.io/projected/c947590c-8931-41a1-b313-183c2d23d223-kube-api-access-bwvqs\") pod \"monitoring-plugin-d5dcff7cf-knm4n\" (UID: \"c947590c-8931-41a1-b313-183c2d23d223\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:04.246450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.246425 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c947590c-8931-41a1-b313-183c2d23d223-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-knm4n\" (UID: \"c947590c-8931-41a1-b313-183c2d23d223\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:04.255880 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.255849 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwvqs\" (UniqueName: \"kubernetes.io/projected/c947590c-8931-41a1-b313-183c2d23d223-kube-api-access-bwvqs\") pod \"monitoring-plugin-d5dcff7cf-knm4n\" (UID: \"c947590c-8931-41a1-b313-183c2d23d223\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:04.326419 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.326323 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:04.470389 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.470290 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n"] Jul 08 19:29:04.473109 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:29:04.473078 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc947590c_8931_41a1_b313_183c2d23d223.slice/crio-73c4ae51e30d6da847b7337575d75bbff74f3f9afa331885ec2cece1a6dd07f3 WatchSource:0}: Error finding container 73c4ae51e30d6da847b7337575d75bbff74f3f9afa331885ec2cece1a6dd07f3: Status 404 returned error can't find the container with id 73c4ae51e30d6da847b7337575d75bbff74f3f9afa331885ec2cece1a6dd07f3 Jul 08 19:29:04.512675 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.512645 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69b5945578-mqtsv"] Jul 08 19:29:04.516886 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.516868 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.519349 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.519324 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 19:29:04.519883 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.519863 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 19:29:04.520159 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.520139 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 19:29:04.520355 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.520329 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 19:29:04.520434 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.520374 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-47jv5\"" Jul 08 19:29:04.520434 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.520418 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 19:29:04.528135 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.528091 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 19:29:04.534804 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.534760 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69b5945578-mqtsv"] Jul 08 19:29:04.578545 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.578456 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" event={"ID":"c947590c-8931-41a1-b313-183c2d23d223","Type":"ContainerStarted","Data":"73c4ae51e30d6da847b7337575d75bbff74f3f9afa331885ec2cece1a6dd07f3"} Jul 08 19:29:04.647088 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.647045 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-federate-client-tls\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.647295 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.647096 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-serving-certs-ca-bundle\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.647295 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.647221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.647295 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.647265 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-secret-telemeter-client\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.647470 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.647335 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.647470 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.647391 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbwcd\" (UniqueName: \"kubernetes.io/projected/ce16abff-9081-4b84-9614-16efb6927f98-kube-api-access-cbwcd\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.647470 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.647455 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-metrics-client-ca\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.647596 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.647492 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-telemeter-client-tls\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.748647 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.748605 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-telemeter-client-tls\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.748824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.748657 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-federate-client-tls\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.748824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.748687 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-serving-certs-ca-bundle\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.748824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.748734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.748824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.748763 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-secret-telemeter-client\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.748824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.748785 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.748824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.748815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbwcd\" (UniqueName: \"kubernetes.io/projected/ce16abff-9081-4b84-9614-16efb6927f98-kube-api-access-cbwcd\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.749155 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.748860 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-metrics-client-ca\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.749646 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.749565 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-serving-certs-ca-bundle\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.749762 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.749641 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-metrics-client-ca\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.750316 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.750263 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ce16abff-9081-4b84-9614-16efb6927f98-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.751838 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.751813 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-federate-client-tls\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.751956 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.751855 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-telemeter-client-tls\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.751956 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.751855 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.752370 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.752340 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ce16abff-9081-4b84-9614-16efb6927f98-secret-telemeter-client\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.757408 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.757388 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbwcd\" (UniqueName: \"kubernetes.io/projected/ce16abff-9081-4b84-9614-16efb6927f98-kube-api-access-cbwcd\") pod \"telemeter-client-69b5945578-mqtsv\" (UID: \"ce16abff-9081-4b84-9614-16efb6927f98\") " pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.827576 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.827540 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" Jul 08 19:29:04.963428 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:04.963273 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69b5945578-mqtsv"] Jul 08 19:29:04.965977 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:29:04.965950 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce16abff_9081_4b84_9614_16efb6927f98.slice/crio-e95af1938e38a0986700053b8c91445bf695c0aefbf822a53ae55fdc9a4bd562 WatchSource:0}: Error finding container e95af1938e38a0986700053b8c91445bf695c0aefbf822a53ae55fdc9a4bd562: Status 404 returned error can't find the container with id e95af1938e38a0986700053b8c91445bf695c0aefbf822a53ae55fdc9a4bd562 Jul 08 19:29:05.584778 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:05.584702 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" event={"ID":"ce16abff-9081-4b84-9614-16efb6927f98","Type":"ContainerStarted","Data":"e95af1938e38a0986700053b8c91445bf695c0aefbf822a53ae55fdc9a4bd562"} Jul 08 19:29:07.591967 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:07.591931 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" event={"ID":"ce16abff-9081-4b84-9614-16efb6927f98","Type":"ContainerStarted","Data":"ea7b8c3a6f885e293fe8b698dc05d2406b5f5b569ef371b8911d171adcd40bad"} Jul 08 19:29:07.593209 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:07.593183 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" event={"ID":"c947590c-8931-41a1-b313-183c2d23d223","Type":"ContainerStarted","Data":"02f6c3ae4a643812ebf4e5f46fa1b6d7ef7acfa14174bbdfff642cf060ed5fef"} Jul 08 19:29:07.593401 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:07.593381 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:07.598018 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:07.597998 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" Jul 08 19:29:07.611942 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:07.611902 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-knm4n" podStartSLOduration=2.496636368 podStartE2EDuration="4.611886943s" totalImagesPullingTime="2.115250575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:29:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:29:07.611707 +0000 UTC m=+309.269145616" watchObservedRunningTime="2026-07-08 19:29:07.611886943 +0000 UTC m=+309.269325550" Jul 08 19:29:08.599050 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:08.598950 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" event={"ID":"ce16abff-9081-4b84-9614-16efb6927f98","Type":"ContainerStarted","Data":"c8b43b8f2f3609ea7b54821c2fccbb4a8cbddfe6bf3c62deee8ea3a00e9a1f7d"} Jul 08 19:29:08.599050 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:08.599040 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" event={"ID":"ce16abff-9081-4b84-9614-16efb6927f98","Type":"ContainerStarted","Data":"e1a1be1daa9cc33d95576a8a5663f6c0b9f10a7c799a31dbef11435b37e8cdc8"} Jul 08 19:29:08.637738 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:08.637682 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69b5945578-mqtsv" podStartSLOduration=1.392161273 podStartE2EDuration="4.637663596s" totalImagesPullingTime="3.245502323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:29:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:29:08.634217744 +0000 UTC m=+310.291656388" watchObservedRunningTime="2026-07-08 19:29:08.637663596 +0000 UTC m=+310.295102204" Jul 08 19:29:10.562651 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.562609 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:29:10.629924 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.629892 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68c468f4c9-6vwtx"] Jul 08 19:29:10.633165 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.633143 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.636866 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.636844 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 19:29:10.636992 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.636889 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 19:29:10.638411 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.638231 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mnqtp\"" Jul 08 19:29:10.638411 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.638241 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 19:29:10.638411 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.638291 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 19:29:10.638411 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.638330 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 19:29:10.655789 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.655758 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68c468f4c9-6vwtx"] Jul 08 19:29:10.704699 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.704663 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-oauth-config\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.704879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.704709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-serving-cert\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.704879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.704786 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-oauth-serving-cert\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.704879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.704834 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-config\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.705049 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.704884 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-service-ca\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.705049 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.704926 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbd2w\" (UniqueName: \"kubernetes.io/projected/4a5c9d2f-3400-47a1-9c62-4281b73e6504-kube-api-access-dbd2w\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.805342 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.805305 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-oauth-serving-cert\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.805342 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.805341 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-config\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.805588 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.805371 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-service-ca\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.805588 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.805398 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbd2w\" (UniqueName: \"kubernetes.io/projected/4a5c9d2f-3400-47a1-9c62-4281b73e6504-kube-api-access-dbd2w\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.805588 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.805432 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-oauth-config\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.805588 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.805451 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-serving-cert\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.806172 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.806150 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-service-ca\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.806297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.806189 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-oauth-serving-cert\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.806367 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.806327 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-config\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.807858 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.807840 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-oauth-config\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.807919 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.807903 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-serving-cert\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.820384 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.820329 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbd2w\" (UniqueName: \"kubernetes.io/projected/4a5c9d2f-3400-47a1-9c62-4281b73e6504-kube-api-access-dbd2w\") pod \"console-68c468f4c9-6vwtx\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:10.943762 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:10.943720 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:11.094886 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:11.094796 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68c468f4c9-6vwtx"] Jul 08 19:29:11.099945 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:29:11.099918 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5c9d2f_3400_47a1_9c62_4281b73e6504.slice/crio-1181582d88585e860629871449a13cf2f9e3a5a2a113f737e1a0ea8776aa37d3 WatchSource:0}: Error finding container 1181582d88585e860629871449a13cf2f9e3a5a2a113f737e1a0ea8776aa37d3: Status 404 returned error can't find the container with id 1181582d88585e860629871449a13cf2f9e3a5a2a113f737e1a0ea8776aa37d3 Jul 08 19:29:11.614801 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:11.614759 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c468f4c9-6vwtx" event={"ID":"4a5c9d2f-3400-47a1-9c62-4281b73e6504","Type":"ContainerStarted","Data":"1181582d88585e860629871449a13cf2f9e3a5a2a113f737e1a0ea8776aa37d3"} Jul 08 19:29:14.628319 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:14.628279 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c468f4c9-6vwtx" event={"ID":"4a5c9d2f-3400-47a1-9c62-4281b73e6504","Type":"ContainerStarted","Data":"5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545"} Jul 08 19:29:14.653641 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:14.653581 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68c468f4c9-6vwtx" podStartSLOduration=1.567902083 podStartE2EDuration="4.653559197s" totalImagesPullingTime="3.085657114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:29:14.651447762 +0000 UTC m=+316.308886372" watchObservedRunningTime="2026-07-08 19:29:14.653559197 +0000 UTC m=+316.310997782" Jul 08 19:29:19.727868 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.727832 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5566777994-vdtmk"] Jul 08 19:29:19.732199 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.732171 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.743674 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.743645 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 19:29:19.749807 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.749782 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5566777994-vdtmk"] Jul 08 19:29:19.883675 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.883640 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-oauth-serving-cert\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.883872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.883692 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-oauth-config\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.883872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.883720 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-serving-cert\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.883872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.883740 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-trusted-ca-bundle\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.883872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.883845 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-service-ca\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.884080 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.883887 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t84kn\" (UniqueName: \"kubernetes.io/projected/ed3c61bb-e32a-4831-9ca9-beda96302d9a-kube-api-access-t84kn\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.884080 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.883918 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-config\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985069 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.984967 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-service-ca\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985069 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985037 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t84kn\" (UniqueName: \"kubernetes.io/projected/ed3c61bb-e32a-4831-9ca9-beda96302d9a-kube-api-access-t84kn\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985069 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985067 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-config\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985340 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985149 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-oauth-serving-cert\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985340 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985171 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-oauth-config\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985340 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-serving-cert\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985340 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985270 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-trusted-ca-bundle\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985837 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985807 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-service-ca\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.985988 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985848 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-config\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.986084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.985985 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-oauth-serving-cert\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.986224 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.986194 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-trusted-ca-bundle\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.987963 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.987943 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-oauth-config\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.988209 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.988185 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-serving-cert\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:19.993923 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:19.993897 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t84kn\" (UniqueName: \"kubernetes.io/projected/ed3c61bb-e32a-4831-9ca9-beda96302d9a-kube-api-access-t84kn\") pod \"console-5566777994-vdtmk\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:20.042834 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:20.042780 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:20.944000 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:20.943956 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:20.944484 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:20.944046 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:20.949659 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:20.949629 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:21.659134 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:21.659104 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:24.178754 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:24.178314 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5566777994-vdtmk"] Jul 08 19:29:24.181413 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:29:24.181386 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3c61bb_e32a_4831_9ca9_beda96302d9a.slice/crio-a7c6d9dc76ad3f94ccd68604657503542f9e4c4f9882385f0e607da7e37cca56 WatchSource:0}: Error finding container a7c6d9dc76ad3f94ccd68604657503542f9e4c4f9882385f0e607da7e37cca56: Status 404 returned error can't find the container with id a7c6d9dc76ad3f94ccd68604657503542f9e4c4f9882385f0e607da7e37cca56 Jul 08 19:29:24.667143 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:24.667041 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-8p4w4" event={"ID":"d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a","Type":"ContainerStarted","Data":"e3b97ee637535f66ed78d56edd8ff9112f17a40e73359db039351efd009e1c91"} Jul 08 19:29:24.667143 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:24.667115 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-8p4w4" Jul 08 19:29:24.669210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:24.669176 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5566777994-vdtmk" event={"ID":"ed3c61bb-e32a-4831-9ca9-beda96302d9a","Type":"ContainerStarted","Data":"2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa"} Jul 08 19:29:24.669341 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:24.669226 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5566777994-vdtmk" event={"ID":"ed3c61bb-e32a-4831-9ca9-beda96302d9a","Type":"ContainerStarted","Data":"a7c6d9dc76ad3f94ccd68604657503542f9e4c4f9882385f0e607da7e37cca56"} Jul 08 19:29:24.679299 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:24.679265 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-8p4w4" Jul 08 19:29:24.685921 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:24.685867 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-8p4w4" podStartSLOduration=1.408984942 podStartE2EDuration="23.68584846s" totalImagesPullingTime="22.276863518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:29:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:29:24.683773498 +0000 UTC m=+326.341212130" watchObservedRunningTime="2026-07-08 19:29:24.68584846 +0000 UTC m=+326.343287067" Jul 08 19:29:24.720177 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:24.719995 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5566777994-vdtmk" podStartSLOduration=5.719954607 podStartE2EDuration="5.719954607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:29:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:29:24.718995176 +0000 UTC m=+326.376433827" watchObservedRunningTime="2026-07-08 19:29:24.719954607 +0000 UTC m=+326.377393214" Jul 08 19:29:30.042960 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:30.042910 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:30.043530 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:30.043036 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:30.048663 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:30.048634 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:30.696499 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:30.696470 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:29:30.746555 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:30.746518 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68c468f4c9-6vwtx"] Jul 08 19:29:37.062126 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:37.062082 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" podUID="11b4adb4-10ba-4735-8fef-6c5560ff821f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 19:29:44.549504 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:44.549474 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6x7mh_bbad0080-35fb-42b3-883d-f81cb83f7789/serve-healthcheck-canary/0.log" Jul 08 19:29:47.061833 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:47.061793 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" podUID="11b4adb4-10ba-4735-8fef-6c5560ff821f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 19:29:55.774609 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:55.774492 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68c468f4c9-6vwtx" podUID="4a5c9d2f-3400-47a1-9c62-4281b73e6504" containerName="console" containerID="cri-o://5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545" gracePeriod=15 Jul 08 19:29:56.089807 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.089776 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:56.177497 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.177466 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-serving-cert\") pod \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " Jul 08 19:29:56.177692 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.177517 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbd2w\" (UniqueName: \"kubernetes.io/projected/4a5c9d2f-3400-47a1-9c62-4281b73e6504-kube-api-access-dbd2w\") pod \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " Jul 08 19:29:56.177692 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.177568 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-oauth-serving-cert\") pod \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " Jul 08 19:29:56.177692 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.177612 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-service-ca\") pod \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " Jul 08 19:29:56.177692 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.177638 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-config\") pod \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " Jul 08 19:29:56.177890 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.177694 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-oauth-config\") pod \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\" (UID: \"4a5c9d2f-3400-47a1-9c62-4281b73e6504\") " Jul 08 19:29:56.178002 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.177945 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a5c9d2f-3400-47a1-9c62-4281b73e6504" (UID: "4a5c9d2f-3400-47a1-9c62-4281b73e6504"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:29:56.178180 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.178154 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a5c9d2f-3400-47a1-9c62-4281b73e6504" (UID: "4a5c9d2f-3400-47a1-9c62-4281b73e6504"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:29:56.178259 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.178203 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-config" (OuterVolumeSpecName: "console-config") pod "4a5c9d2f-3400-47a1-9c62-4281b73e6504" (UID: "4a5c9d2f-3400-47a1-9c62-4281b73e6504"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:29:56.179901 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.179853 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a5c9d2f-3400-47a1-9c62-4281b73e6504" (UID: "4a5c9d2f-3400-47a1-9c62-4281b73e6504"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:29:56.180012 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.179941 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a5c9d2f-3400-47a1-9c62-4281b73e6504-kube-api-access-dbd2w" (OuterVolumeSpecName: "kube-api-access-dbd2w") pod "4a5c9d2f-3400-47a1-9c62-4281b73e6504" (UID: "4a5c9d2f-3400-47a1-9c62-4281b73e6504"). InnerVolumeSpecName "kube-api-access-dbd2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:29:56.180012 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.179984 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a5c9d2f-3400-47a1-9c62-4281b73e6504" (UID: "4a5c9d2f-3400-47a1-9c62-4281b73e6504"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:29:56.279045 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.278874 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-service-ca\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:29:56.279045 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.278913 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-config\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:29:56.279045 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.278928 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-oauth-config\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:29:56.279045 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.278946 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a5c9d2f-3400-47a1-9c62-4281b73e6504-console-serving-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:29:56.279045 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.278961 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbd2w\" (UniqueName: \"kubernetes.io/projected/4a5c9d2f-3400-47a1-9c62-4281b73e6504-kube-api-access-dbd2w\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:29:56.279045 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.278978 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a5c9d2f-3400-47a1-9c62-4281b73e6504-oauth-serving-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:29:56.784050 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.783991 2573 generic.go:356] "Generic (PLEG): container finished" podID="4a5c9d2f-3400-47a1-9c62-4281b73e6504" containerID="5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545" exitCode=0 Jul 08 19:29:56.784539 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.784076 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c468f4c9-6vwtx" event={"ID":"4a5c9d2f-3400-47a1-9c62-4281b73e6504","Type":"ContainerDied","Data":"5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545"} Jul 08 19:29:56.784539 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.784133 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c468f4c9-6vwtx" event={"ID":"4a5c9d2f-3400-47a1-9c62-4281b73e6504","Type":"ContainerDied","Data":"1181582d88585e860629871449a13cf2f9e3a5a2a113f737e1a0ea8776aa37d3"} Jul 08 19:29:56.784539 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.784163 2573 scope.go:117] "RemoveContainer" containerID="5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545" Jul 08 19:29:56.784539 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.784341 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c468f4c9-6vwtx" Jul 08 19:29:56.801318 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.801284 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68c468f4c9-6vwtx"] Jul 08 19:29:56.802781 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.802760 2573 scope.go:117] "RemoveContainer" containerID="5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545" Jul 08 19:29:56.803134 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:29:56.803090 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545\": container with ID starting with 5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545 not found: ID does not exist" containerID="5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545" Jul 08 19:29:56.803261 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.803133 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545"} err="failed to get container status \"5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545\": rpc error: code = NotFound desc = could not find container \"5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545\": container with ID starting with 5c384f672ef4875b8e6805f717d62b0f66c75728bca5dc89e0a73b371a8b6545 not found: ID does not exist" Jul 08 19:29:56.805581 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:56.805557 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68c468f4c9-6vwtx"] Jul 08 19:29:57.064858 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:57.064796 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" podUID="11b4adb4-10ba-4735-8fef-6c5560ff821f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 08 19:29:57.065100 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:57.064910 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" Jul 08 19:29:57.065638 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:57.065605 2573 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"1a8c042ce5f486edbbb747a1e6ad2e4c96121937b818ba7b6079d97d26866ba8"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 08 19:29:57.065742 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:57.065661 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" podUID="11b4adb4-10ba-4735-8fef-6c5560ff821f" containerName="service-proxy" containerID="cri-o://1a8c042ce5f486edbbb747a1e6ad2e4c96121937b818ba7b6079d97d26866ba8" gracePeriod=30 Jul 08 19:29:57.790158 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:57.790123 2573 generic.go:356] "Generic (PLEG): container finished" podID="11b4adb4-10ba-4735-8fef-6c5560ff821f" containerID="1a8c042ce5f486edbbb747a1e6ad2e4c96121937b818ba7b6079d97d26866ba8" exitCode=2 Jul 08 19:29:57.790589 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:57.790193 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" event={"ID":"11b4adb4-10ba-4735-8fef-6c5560ff821f","Type":"ContainerDied","Data":"1a8c042ce5f486edbbb747a1e6ad2e4c96121937b818ba7b6079d97d26866ba8"} Jul 08 19:29:57.790589 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:57.790239 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-656566c579-gw9pd" event={"ID":"11b4adb4-10ba-4735-8fef-6c5560ff821f","Type":"ContainerStarted","Data":"8f3be0f3a71f8b03f358e93b767eb0f05482f31e9e3d9bb294a18d67211dc0de"} Jul 08 19:29:58.567262 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:29:58.567229 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a5c9d2f-3400-47a1-9c62-4281b73e6504" path="/var/lib/kubelet/pods/4a5c9d2f-3400-47a1-9c62-4281b73e6504/volumes" Jul 08 19:30:14.323420 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:14.323379 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:30:14.325859 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:14.325834 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/582ae813-01ce-4fef-86ea-6dfc6ea19f3d-metrics-certs\") pod \"network-metrics-daemon-ndsk6\" (UID: \"582ae813-01ce-4fef-86ea-6dfc6ea19f3d\") " pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:30:14.465817 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:14.465787 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgx5c\"" Jul 08 19:30:14.473866 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:14.473845 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndsk6" Jul 08 19:30:14.607010 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:14.606850 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ndsk6"] Jul 08 19:30:14.609451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:30:14.609414 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582ae813_01ce_4fef_86ea_6dfc6ea19f3d.slice/crio-e21e6c7580c9051cedaace54316a69a4c5f9ebf1ec95468e88484f346e99bad4 WatchSource:0}: Error finding container e21e6c7580c9051cedaace54316a69a4c5f9ebf1ec95468e88484f346e99bad4: Status 404 returned error can't find the container with id e21e6c7580c9051cedaace54316a69a4c5f9ebf1ec95468e88484f346e99bad4 Jul 08 19:30:14.853142 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:14.853102 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndsk6" event={"ID":"582ae813-01ce-4fef-86ea-6dfc6ea19f3d","Type":"ContainerStarted","Data":"e21e6c7580c9051cedaace54316a69a4c5f9ebf1ec95468e88484f346e99bad4"} Jul 08 19:30:15.857769 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:15.857731 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndsk6" event={"ID":"582ae813-01ce-4fef-86ea-6dfc6ea19f3d","Type":"ContainerStarted","Data":"63d8f9330b9415f5978af25522644dc7fefd145deb6a57eef0a8d1f2f4ef7d08"} Jul 08 19:30:16.382289 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.382258 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7897b8dd64-5d78v"] Jul 08 19:30:16.382508 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.382495 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a5c9d2f-3400-47a1-9c62-4281b73e6504" containerName="console" Jul 08 19:30:16.382508 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.382508 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a5c9d2f-3400-47a1-9c62-4281b73e6504" containerName="console" Jul 08 19:30:16.382592 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.382558 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a5c9d2f-3400-47a1-9c62-4281b73e6504" containerName="console" Jul 08 19:30:16.384426 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.384411 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.395136 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.395102 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7897b8dd64-5d78v"] Jul 08 19:30:16.541104 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.541063 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-service-ca\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.541280 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.541122 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvhw8\" (UniqueName: \"kubernetes.io/projected/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-kube-api-access-cvhw8\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.541280 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.541197 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-config\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.541280 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.541231 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-oauth-serving-cert\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.541280 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.541264 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-serving-cert\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.541437 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.541285 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-trusted-ca-bundle\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.541437 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.541334 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-oauth-config\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.642554 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.642451 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvhw8\" (UniqueName: \"kubernetes.io/projected/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-kube-api-access-cvhw8\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.642554 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.642520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-config\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.642554 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.642546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-oauth-serving-cert\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.642853 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.642578 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-serving-cert\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.642853 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.642601 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-trusted-ca-bundle\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.642853 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.642628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-oauth-config\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.642853 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.642668 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-service-ca\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.643438 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.643399 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-config\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.643438 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.643399 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-oauth-serving-cert\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.643619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.643455 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-service-ca\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.643619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.643522 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-trusted-ca-bundle\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.645249 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.645221 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-oauth-config\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.645338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.645306 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-serving-cert\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.653133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.653107 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvhw8\" (UniqueName: \"kubernetes.io/projected/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-kube-api-access-cvhw8\") pod \"console-7897b8dd64-5d78v\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.693147 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.693113 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:16.824871 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.824831 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7897b8dd64-5d78v"] Jul 08 19:30:16.828061 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:30:16.828019 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16bc0ff_e5f9_487a_8aa4_a827ed7a82c5.slice/crio-4c44ee452248ed0c24010669da1f36dfb5c473b89bcd2b5f43c444e289e66be9 WatchSource:0}: Error finding container 4c44ee452248ed0c24010669da1f36dfb5c473b89bcd2b5f43c444e289e66be9: Status 404 returned error can't find the container with id 4c44ee452248ed0c24010669da1f36dfb5c473b89bcd2b5f43c444e289e66be9 Jul 08 19:30:16.864094 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.864064 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndsk6" event={"ID":"582ae813-01ce-4fef-86ea-6dfc6ea19f3d","Type":"ContainerStarted","Data":"5c1e79f9ec361895954d66a73a40ce43548459cf709567db6e72b4b77bd27565"} Jul 08 19:30:16.865504 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.865473 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7897b8dd64-5d78v" event={"ID":"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5","Type":"ContainerStarted","Data":"4c44ee452248ed0c24010669da1f36dfb5c473b89bcd2b5f43c444e289e66be9"} Jul 08 19:30:16.888801 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:16.888753 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ndsk6" podStartSLOduration=374.781779046 podStartE2EDuration="6m15.888737299s" totalImagesPullingTime="1.106958253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:30:16.887101851 +0000 UTC m=+378.544540476" watchObservedRunningTime="2026-07-08 19:30:16.888737299 +0000 UTC m=+378.546175914" Jul 08 19:30:17.870978 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:17.870935 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7897b8dd64-5d78v" event={"ID":"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5","Type":"ContainerStarted","Data":"e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78"} Jul 08 19:30:17.893778 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:17.893725 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7897b8dd64-5d78v" podStartSLOduration=1.89370393 podStartE2EDuration="1.89370393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:30:17.893192456 +0000 UTC m=+379.550631105" watchObservedRunningTime="2026-07-08 19:30:17.89370393 +0000 UTC m=+379.551142537" Jul 08 19:30:26.693785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:26.693739 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:26.694251 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:26.693816 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:26.698414 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:26.698389 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:26.906019 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:26.905988 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:30:26.952168 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:26.952089 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5566777994-vdtmk"] Jul 08 19:30:51.971805 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:51.971738 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5566777994-vdtmk" podUID="ed3c61bb-e32a-4831-9ca9-beda96302d9a" containerName="console" containerID="cri-o://2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa" gracePeriod=15 Jul 08 19:30:52.215252 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.215226 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:30:52.304752 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.304716 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-oauth-config\") pod \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " Jul 08 19:30:52.304942 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.304767 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-trusted-ca-bundle\") pod \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " Jul 08 19:30:52.304942 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.304799 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-service-ca\") pod \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " Jul 08 19:30:52.304942 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.304819 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-config\") pod \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " Jul 08 19:30:52.304942 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.304845 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-oauth-serving-cert\") pod \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " Jul 08 19:30:52.304942 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.304863 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-serving-cert\") pod \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " Jul 08 19:30:52.304942 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.304880 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t84kn\" (UniqueName: \"kubernetes.io/projected/ed3c61bb-e32a-4831-9ca9-beda96302d9a-kube-api-access-t84kn\") pod \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\" (UID: \"ed3c61bb-e32a-4831-9ca9-beda96302d9a\") " Jul 08 19:30:52.305330 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.305304 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-config" (OuterVolumeSpecName: "console-config") pod "ed3c61bb-e32a-4831-9ca9-beda96302d9a" (UID: "ed3c61bb-e32a-4831-9ca9-beda96302d9a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:30:52.305395 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.305321 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-service-ca" (OuterVolumeSpecName: "service-ca") pod "ed3c61bb-e32a-4831-9ca9-beda96302d9a" (UID: "ed3c61bb-e32a-4831-9ca9-beda96302d9a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:30:52.305395 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.305329 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ed3c61bb-e32a-4831-9ca9-beda96302d9a" (UID: "ed3c61bb-e32a-4831-9ca9-beda96302d9a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:30:52.305481 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.305394 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ed3c61bb-e32a-4831-9ca9-beda96302d9a" (UID: "ed3c61bb-e32a-4831-9ca9-beda96302d9a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:30:52.306914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.306888 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ed3c61bb-e32a-4831-9ca9-beda96302d9a" (UID: "ed3c61bb-e32a-4831-9ca9-beda96302d9a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:30:52.307145 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.307125 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ed3c61bb-e32a-4831-9ca9-beda96302d9a" (UID: "ed3c61bb-e32a-4831-9ca9-beda96302d9a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:30:52.307217 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.307204 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3c61bb-e32a-4831-9ca9-beda96302d9a-kube-api-access-t84kn" (OuterVolumeSpecName: "kube-api-access-t84kn") pod "ed3c61bb-e32a-4831-9ca9-beda96302d9a" (UID: "ed3c61bb-e32a-4831-9ca9-beda96302d9a"). InnerVolumeSpecName "kube-api-access-t84kn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:30:52.406359 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.406320 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-oauth-serving-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:30:52.406359 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.406350 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-serving-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:30:52.406359 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.406359 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t84kn\" (UniqueName: \"kubernetes.io/projected/ed3c61bb-e32a-4831-9ca9-beda96302d9a-kube-api-access-t84kn\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:30:52.406359 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.406368 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-oauth-config\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:30:52.406604 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.406376 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-trusted-ca-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:30:52.406604 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.406386 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-service-ca\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:30:52.406604 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:52.406394 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ed3c61bb-e32a-4831-9ca9-beda96302d9a-console-config\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:30:53.000208 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.000176 2573 generic.go:356] "Generic (PLEG): container finished" podID="ed3c61bb-e32a-4831-9ca9-beda96302d9a" containerID="2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa" exitCode=0 Jul 08 19:30:53.000638 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.000223 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5566777994-vdtmk" Jul 08 19:30:53.000638 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.000236 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5566777994-vdtmk" event={"ID":"ed3c61bb-e32a-4831-9ca9-beda96302d9a","Type":"ContainerDied","Data":"2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa"} Jul 08 19:30:53.000638 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.000276 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5566777994-vdtmk" event={"ID":"ed3c61bb-e32a-4831-9ca9-beda96302d9a","Type":"ContainerDied","Data":"a7c6d9dc76ad3f94ccd68604657503542f9e4c4f9882385f0e607da7e37cca56"} Jul 08 19:30:53.000638 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.000295 2573 scope.go:117] "RemoveContainer" containerID="2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa" Jul 08 19:30:53.007738 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.007720 2573 scope.go:117] "RemoveContainer" containerID="2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa" Jul 08 19:30:53.008005 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:30:53.007985 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa\": container with ID starting with 2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa not found: ID does not exist" containerID="2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa" Jul 08 19:30:53.008084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.008013 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa"} err="failed to get container status \"2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa\": rpc error: code = NotFound desc = could not find container \"2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa\": container with ID starting with 2a65a27679294d1b9be2cfb655309182c78d64e9a4d32bc1f51871858c588caa not found: ID does not exist" Jul 08 19:30:53.025820 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.025794 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5566777994-vdtmk"] Jul 08 19:30:53.034145 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:53.034124 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5566777994-vdtmk"] Jul 08 19:30:54.568994 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:30:54.568956 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed3c61bb-e32a-4831-9ca9-beda96302d9a" path="/var/lib/kubelet/pods/ed3c61bb-e32a-4831-9ca9-beda96302d9a/volumes" Jul 08 19:31:13.021007 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.020969 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf"] Jul 08 19:31:13.021423 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.021213 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3c61bb-e32a-4831-9ca9-beda96302d9a" containerName="console" Jul 08 19:31:13.021423 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.021225 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3c61bb-e32a-4831-9ca9-beda96302d9a" containerName="console" Jul 08 19:31:13.021423 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.021280 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3c61bb-e32a-4831-9ca9-beda96302d9a" containerName="console" Jul 08 19:31:13.024369 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.024352 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.026717 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.026693 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2jsr5\"" Jul 08 19:31:13.026827 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.026805 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 19:31:13.027670 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.027653 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 19:31:13.033268 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.033246 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf"] Jul 08 19:31:13.158204 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.158164 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnw2s\" (UniqueName: \"kubernetes.io/projected/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-kube-api-access-qnw2s\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.158204 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.158206 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.158415 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.158312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.259124 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.259072 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.259124 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.259134 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnw2s\" (UniqueName: \"kubernetes.io/projected/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-kube-api-access-qnw2s\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.259308 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.259159 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.259547 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.259530 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.259583 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.259528 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.267296 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.267270 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnw2s\" (UniqueName: \"kubernetes.io/projected/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-kube-api-access-qnw2s\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.333485 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.333388 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:13.461134 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:13.461104 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf"] Jul 08 19:31:13.464639 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:31:13.464613 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a52ef6d_13ae_4ab6_80e5_f7962ea40c2b.slice/crio-96ff77a72bbdd366e3d7a01a804525159eefdde3437f26548bdeb2fad7619242 WatchSource:0}: Error finding container 96ff77a72bbdd366e3d7a01a804525159eefdde3437f26548bdeb2fad7619242: Status 404 returned error can't find the container with id 96ff77a72bbdd366e3d7a01a804525159eefdde3437f26548bdeb2fad7619242 Jul 08 19:31:14.076137 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:14.076099 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" event={"ID":"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b","Type":"ContainerStarted","Data":"96ff77a72bbdd366e3d7a01a804525159eefdde3437f26548bdeb2fad7619242"} Jul 08 19:31:26.120237 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:26.120153 2573 generic.go:356] "Generic (PLEG): container finished" podID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerID="3fdf003bdcc3cb9fcb5a082ed9e1af179a3892e7bebed0a69a4bf77e7eab7105" exitCode=0 Jul 08 19:31:26.120606 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:26.120243 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" event={"ID":"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b","Type":"ContainerDied","Data":"3fdf003bdcc3cb9fcb5a082ed9e1af179a3892e7bebed0a69a4bf77e7eab7105"} Jul 08 19:31:28.130810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:28.130765 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" event={"ID":"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b","Type":"ContainerStarted","Data":"356dedc9dfd4b1ab250fbe8f44384d974b3623ab056e3a2e88bc274e6f55a104"} Jul 08 19:31:29.136210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:29.136175 2573 generic.go:356] "Generic (PLEG): container finished" podID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerID="356dedc9dfd4b1ab250fbe8f44384d974b3623ab056e3a2e88bc274e6f55a104" exitCode=0 Jul 08 19:31:29.136599 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:29.136229 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" event={"ID":"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b","Type":"ContainerDied","Data":"356dedc9dfd4b1ab250fbe8f44384d974b3623ab056e3a2e88bc274e6f55a104"} Jul 08 19:31:36.165062 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:36.165005 2573 generic.go:356] "Generic (PLEG): container finished" podID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerID="38506517fb6a561455666fda7ddf4b65e7405b5b76a05880743b7748b7b7c2df" exitCode=0 Jul 08 19:31:36.165443 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:36.165086 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" event={"ID":"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b","Type":"ContainerDied","Data":"38506517fb6a561455666fda7ddf4b65e7405b5b76a05880743b7748b7b7c2df"} Jul 08 19:31:37.288836 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.288811 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:37.456704 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.456587 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-bundle\") pod \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " Jul 08 19:31:37.456704 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.456670 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnw2s\" (UniqueName: \"kubernetes.io/projected/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-kube-api-access-qnw2s\") pod \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " Jul 08 19:31:37.456933 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.456722 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-util\") pod \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\" (UID: \"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b\") " Jul 08 19:31:37.457367 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.457330 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-bundle" (OuterVolumeSpecName: "bundle") pod "0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" (UID: "0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:31:37.458839 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.458815 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-kube-api-access-qnw2s" (OuterVolumeSpecName: "kube-api-access-qnw2s") pod "0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" (UID: "0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b"). InnerVolumeSpecName "kube-api-access-qnw2s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:31:37.462177 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.462146 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-util" (OuterVolumeSpecName: "util") pod "0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" (UID: "0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:31:37.557799 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.557743 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-util\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:31:37.557799 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.557791 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:31:37.557799 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:37.557804 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnw2s\" (UniqueName: \"kubernetes.io/projected/0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b-kube-api-access-qnw2s\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:31:38.174118 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:38.174088 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" Jul 08 19:31:38.174282 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:38.174089 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rm4gf" event={"ID":"0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b","Type":"ContainerDied","Data":"96ff77a72bbdd366e3d7a01a804525159eefdde3437f26548bdeb2fad7619242"} Jul 08 19:31:38.174282 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:38.174209 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96ff77a72bbdd366e3d7a01a804525159eefdde3437f26548bdeb2fad7619242" Jul 08 19:31:45.898185 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.898150 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m"] Jul 08 19:31:45.898637 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.898427 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerName="util" Jul 08 19:31:45.898637 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.898438 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerName="util" Jul 08 19:31:45.898637 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.898452 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerName="pull" Jul 08 19:31:45.898637 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.898458 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerName="pull" Jul 08 19:31:45.898637 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.898463 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerName="extract" Jul 08 19:31:45.898637 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.898469 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerName="extract" Jul 08 19:31:45.898637 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.898518 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a52ef6d-13ae-4ab6-80e5-f7962ea40c2b" containerName="extract" Jul 08 19:31:45.902186 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.902169 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" Jul 08 19:31:45.904505 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.904483 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:31:45.904643 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.904526 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 19:31:45.904712 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.904665 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-69hth\"" Jul 08 19:31:45.913801 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:45.913778 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m"] Jul 08 19:31:46.020850 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:46.020811 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqldc\" (UniqueName: \"kubernetes.io/projected/1e44608a-1834-4f04-986e-aaddab8e849d-kube-api-access-rqldc\") pod \"cert-manager-operator-controller-manager-55c444bf86-n5z6m\" (UID: \"1e44608a-1834-4f04-986e-aaddab8e849d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" Jul 08 19:31:46.021054 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:46.020870 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e44608a-1834-4f04-986e-aaddab8e849d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-n5z6m\" (UID: \"1e44608a-1834-4f04-986e-aaddab8e849d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" Jul 08 19:31:46.121770 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:46.121727 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e44608a-1834-4f04-986e-aaddab8e849d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-n5z6m\" (UID: \"1e44608a-1834-4f04-986e-aaddab8e849d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" Jul 08 19:31:46.121969 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:46.121794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqldc\" (UniqueName: \"kubernetes.io/projected/1e44608a-1834-4f04-986e-aaddab8e849d-kube-api-access-rqldc\") pod \"cert-manager-operator-controller-manager-55c444bf86-n5z6m\" (UID: \"1e44608a-1834-4f04-986e-aaddab8e849d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" Jul 08 19:31:46.122195 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:46.122174 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e44608a-1834-4f04-986e-aaddab8e849d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-n5z6m\" (UID: \"1e44608a-1834-4f04-986e-aaddab8e849d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" Jul 08 19:31:46.132923 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:46.132886 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqldc\" (UniqueName: \"kubernetes.io/projected/1e44608a-1834-4f04-986e-aaddab8e849d-kube-api-access-rqldc\") pod \"cert-manager-operator-controller-manager-55c444bf86-n5z6m\" (UID: \"1e44608a-1834-4f04-986e-aaddab8e849d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" Jul 08 19:31:46.211428 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:46.211340 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" Jul 08 19:31:46.341107 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:46.341074 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m"] Jul 08 19:31:46.344387 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:31:46.344359 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e44608a_1834_4f04_986e_aaddab8e849d.slice/crio-36e4561db2f614995eb85003213b5858d137b9471899b088b07043d58907909b WatchSource:0}: Error finding container 36e4561db2f614995eb85003213b5858d137b9471899b088b07043d58907909b: Status 404 returned error can't find the container with id 36e4561db2f614995eb85003213b5858d137b9471899b088b07043d58907909b Jul 08 19:31:47.208364 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:47.208334 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" event={"ID":"1e44608a-1834-4f04-986e-aaddab8e849d","Type":"ContainerStarted","Data":"36e4561db2f614995eb85003213b5858d137b9471899b088b07043d58907909b"} Jul 08 19:31:48.214253 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:48.214160 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" event={"ID":"1e44608a-1834-4f04-986e-aaddab8e849d","Type":"ContainerStarted","Data":"94aa51d0086ce20622e48d2dbd4d2ebdd9cf1346e537d55b0f39bd5381433319"} Jul 08 19:31:48.235766 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:48.235696 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-n5z6m" podStartSLOduration=1.780412197 podStartE2EDuration="3.235674221s" totalImagesPullingTime="1.455262024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:31:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:31:48.234985407 +0000 UTC m=+469.892424014" watchObservedRunningTime="2026-07-08 19:31:48.235674221 +0000 UTC m=+469.893112828" Jul 08 19:31:49.972255 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:49.972221 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb"] Jul 08 19:31:49.974513 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:49.974497 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:49.976638 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:49.976615 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 19:31:49.977557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:49.977533 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2jsr5\"" Jul 08 19:31:49.977557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:49.977533 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 19:31:49.983113 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:49.983091 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb"] Jul 08 19:31:50.153237 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.153195 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.153237 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.153233 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.153466 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.153258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg4fm\" (UniqueName: \"kubernetes.io/projected/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-kube-api-access-xg4fm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.254646 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.254610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.254818 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.254658 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.254818 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.254706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg4fm\" (UniqueName: \"kubernetes.io/projected/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-kube-api-access-xg4fm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.255001 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.254982 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.255060 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.255018 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.262754 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.262724 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg4fm\" (UniqueName: \"kubernetes.io/projected/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-kube-api-access-xg4fm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.283206 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.283176 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:50.409904 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:50.409867 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb"] Jul 08 19:31:50.413480 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:31:50.413453 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4640cf_3f75_49b2_aa69_abd1ca5f36ec.slice/crio-4b07784b33c9b7e6940e2bab783ce8a3a2341f3e047f5e2f0235faeac231fa95 WatchSource:0}: Error finding container 4b07784b33c9b7e6940e2bab783ce8a3a2341f3e047f5e2f0235faeac231fa95: Status 404 returned error can't find the container with id 4b07784b33c9b7e6940e2bab783ce8a3a2341f3e047f5e2f0235faeac231fa95 Jul 08 19:31:51.230554 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:51.230518 2573 generic.go:356] "Generic (PLEG): container finished" podID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerID="b785a9ea495d2b8aa1d7ab87275b330e500244ddd007ed000b6db14459aff350" exitCode=0 Jul 08 19:31:51.230554 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:51.230559 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" event={"ID":"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec","Type":"ContainerDied","Data":"b785a9ea495d2b8aa1d7ab87275b330e500244ddd007ed000b6db14459aff350"} Jul 08 19:31:51.230987 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:51.230601 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" event={"ID":"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec","Type":"ContainerStarted","Data":"4b07784b33c9b7e6940e2bab783ce8a3a2341f3e047f5e2f0235faeac231fa95"} Jul 08 19:31:54.246611 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:54.246578 2573 generic.go:356] "Generic (PLEG): container finished" podID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerID="142dc209c2a54455f5e8f48f898c71e1aac453e7cf421a7bb498f06ad09ca7e0" exitCode=0 Jul 08 19:31:54.247098 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:54.246651 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" event={"ID":"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec","Type":"ContainerDied","Data":"142dc209c2a54455f5e8f48f898c71e1aac453e7cf421a7bb498f06ad09ca7e0"} Jul 08 19:31:55.252726 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:55.252692 2573 generic.go:356] "Generic (PLEG): container finished" podID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerID="3aaa79aed1cfc0d21ea1a844ce419aed52eb8e3616311793b5766779b7984e58" exitCode=0 Jul 08 19:31:55.253131 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:55.252785 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" event={"ID":"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec","Type":"ContainerDied","Data":"3aaa79aed1cfc0d21ea1a844ce419aed52eb8e3616311793b5766779b7984e58"} Jul 08 19:31:56.381608 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.381585 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:56.398854 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.398826 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-util\") pod \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " Jul 08 19:31:56.398987 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.398875 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-bundle\") pod \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " Jul 08 19:31:56.398987 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.398969 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg4fm\" (UniqueName: \"kubernetes.io/projected/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-kube-api-access-xg4fm\") pod \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\" (UID: \"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec\") " Jul 08 19:31:56.399257 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.399234 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-bundle" (OuterVolumeSpecName: "bundle") pod "cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" (UID: "cf4640cf-3f75-49b2-aa69-abd1ca5f36ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:31:56.401287 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.401243 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-kube-api-access-xg4fm" (OuterVolumeSpecName: "kube-api-access-xg4fm") pod "cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" (UID: "cf4640cf-3f75-49b2-aa69-abd1ca5f36ec"). InnerVolumeSpecName "kube-api-access-xg4fm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:31:56.405449 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.405351 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-util" (OuterVolumeSpecName: "util") pod "cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" (UID: "cf4640cf-3f75-49b2-aa69-abd1ca5f36ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:31:56.499479 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.499424 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg4fm\" (UniqueName: \"kubernetes.io/projected/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-kube-api-access-xg4fm\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:31:56.499479 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.499474 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-util\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:31:56.499479 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:56.499484 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cf4640cf-3f75-49b2-aa69-abd1ca5f36ec-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:31:57.262433 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:57.262396 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" event={"ID":"cf4640cf-3f75-49b2-aa69-abd1ca5f36ec","Type":"ContainerDied","Data":"4b07784b33c9b7e6940e2bab783ce8a3a2341f3e047f5e2f0235faeac231fa95"} Jul 08 19:31:57.262433 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:57.262423 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fd2dbb" Jul 08 19:31:57.262642 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:57.262450 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b07784b33c9b7e6940e2bab783ce8a3a2341f3e047f5e2f0235faeac231fa95" Jul 08 19:31:58.877451 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.877418 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-984z2"] Jul 08 19:31:58.877939 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.877806 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerName="pull" Jul 08 19:31:58.877939 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.877823 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerName="pull" Jul 08 19:31:58.877939 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.877846 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerName="util" Jul 08 19:31:58.877939 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.877853 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerName="util" Jul 08 19:31:58.877939 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.877865 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerName="extract" Jul 08 19:31:58.877939 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.877872 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerName="extract" Jul 08 19:31:58.877939 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.877928 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf4640cf-3f75-49b2-aa69-abd1ca5f36ec" containerName="extract" Jul 08 19:31:58.879821 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.879800 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-984z2" Jul 08 19:31:58.882103 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.882077 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-8bnbk\"" Jul 08 19:31:58.882302 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.882281 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 19:31:58.883116 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.883097 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 19:31:58.887909 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.887885 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-984z2"] Jul 08 19:31:58.919593 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.919560 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05290855-3013-4ad2-9be1-c2ac618bd75a-bound-sa-token\") pod \"cert-manager-55b4d954d5-984z2\" (UID: \"05290855-3013-4ad2-9be1-c2ac618bd75a\") " pod="cert-manager/cert-manager-55b4d954d5-984z2" Jul 08 19:31:58.919764 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:58.919610 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j85cq\" (UniqueName: \"kubernetes.io/projected/05290855-3013-4ad2-9be1-c2ac618bd75a-kube-api-access-j85cq\") pod \"cert-manager-55b4d954d5-984z2\" (UID: \"05290855-3013-4ad2-9be1-c2ac618bd75a\") " pod="cert-manager/cert-manager-55b4d954d5-984z2" Jul 08 19:31:59.020756 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:59.020717 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j85cq\" (UniqueName: \"kubernetes.io/projected/05290855-3013-4ad2-9be1-c2ac618bd75a-kube-api-access-j85cq\") pod \"cert-manager-55b4d954d5-984z2\" (UID: \"05290855-3013-4ad2-9be1-c2ac618bd75a\") " pod="cert-manager/cert-manager-55b4d954d5-984z2" Jul 08 19:31:59.020966 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:59.020848 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05290855-3013-4ad2-9be1-c2ac618bd75a-bound-sa-token\") pod \"cert-manager-55b4d954d5-984z2\" (UID: \"05290855-3013-4ad2-9be1-c2ac618bd75a\") " pod="cert-manager/cert-manager-55b4d954d5-984z2" Jul 08 19:31:59.028494 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:59.028469 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05290855-3013-4ad2-9be1-c2ac618bd75a-bound-sa-token\") pod \"cert-manager-55b4d954d5-984z2\" (UID: \"05290855-3013-4ad2-9be1-c2ac618bd75a\") " pod="cert-manager/cert-manager-55b4d954d5-984z2" Jul 08 19:31:59.028604 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:59.028578 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j85cq\" (UniqueName: \"kubernetes.io/projected/05290855-3013-4ad2-9be1-c2ac618bd75a-kube-api-access-j85cq\") pod \"cert-manager-55b4d954d5-984z2\" (UID: \"05290855-3013-4ad2-9be1-c2ac618bd75a\") " pod="cert-manager/cert-manager-55b4d954d5-984z2" Jul 08 19:31:59.189396 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:59.189309 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-984z2" Jul 08 19:31:59.315519 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:31:59.315494 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-984z2"] Jul 08 19:31:59.317960 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:31:59.317934 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05290855_3013_4ad2_9be1_c2ac618bd75a.slice/crio-d267dcf3e109220253990a03e1aec4ff5b29336a84d4df12bf8de3ab1b062f19 WatchSource:0}: Error finding container d267dcf3e109220253990a03e1aec4ff5b29336a84d4df12bf8de3ab1b062f19: Status 404 returned error can't find the container with id d267dcf3e109220253990a03e1aec4ff5b29336a84d4df12bf8de3ab1b062f19 Jul 08 19:32:00.276616 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:00.276580 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-984z2" event={"ID":"05290855-3013-4ad2-9be1-c2ac618bd75a","Type":"ContainerStarted","Data":"d267dcf3e109220253990a03e1aec4ff5b29336a84d4df12bf8de3ab1b062f19"} Jul 08 19:32:02.601714 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.601632 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q"] Jul 08 19:32:02.603729 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.603712 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" Jul 08 19:32:02.606267 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.606235 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 19:32:02.606370 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.606273 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:32:02.607171 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.607150 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-t56dg\"" Jul 08 19:32:02.615112 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.615089 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q"] Jul 08 19:32:02.650135 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.650099 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbj6w\" (UniqueName: \"kubernetes.io/projected/217f7efe-cd01-4b57-8e98-3d52d6420125-kube-api-access-vbj6w\") pod \"openshift-lws-operator-fd8ccff4c-sgr4q\" (UID: \"217f7efe-cd01-4b57-8e98-3d52d6420125\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" Jul 08 19:32:02.650302 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.650170 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/217f7efe-cd01-4b57-8e98-3d52d6420125-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sgr4q\" (UID: \"217f7efe-cd01-4b57-8e98-3d52d6420125\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" Jul 08 19:32:02.750746 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.750702 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/217f7efe-cd01-4b57-8e98-3d52d6420125-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sgr4q\" (UID: \"217f7efe-cd01-4b57-8e98-3d52d6420125\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" Jul 08 19:32:02.750923 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.750762 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbj6w\" (UniqueName: \"kubernetes.io/projected/217f7efe-cd01-4b57-8e98-3d52d6420125-kube-api-access-vbj6w\") pod \"openshift-lws-operator-fd8ccff4c-sgr4q\" (UID: \"217f7efe-cd01-4b57-8e98-3d52d6420125\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" Jul 08 19:32:02.751219 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.751198 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/217f7efe-cd01-4b57-8e98-3d52d6420125-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sgr4q\" (UID: \"217f7efe-cd01-4b57-8e98-3d52d6420125\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" Jul 08 19:32:02.759104 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.759080 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbj6w\" (UniqueName: \"kubernetes.io/projected/217f7efe-cd01-4b57-8e98-3d52d6420125-kube-api-access-vbj6w\") pod \"openshift-lws-operator-fd8ccff4c-sgr4q\" (UID: \"217f7efe-cd01-4b57-8e98-3d52d6420125\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" Jul 08 19:32:02.912578 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:02.912491 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" Jul 08 19:32:03.054881 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:03.054857 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q"] Jul 08 19:32:03.057445 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:03.057412 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod217f7efe_cd01_4b57_8e98_3d52d6420125.slice/crio-6126da60cc1d9f9ecef6e4622576adc7c431cf12e4de3ca294bc5b68eb4f3961 WatchSource:0}: Error finding container 6126da60cc1d9f9ecef6e4622576adc7c431cf12e4de3ca294bc5b68eb4f3961: Status 404 returned error can't find the container with id 6126da60cc1d9f9ecef6e4622576adc7c431cf12e4de3ca294bc5b68eb4f3961 Jul 08 19:32:03.291744 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:03.291700 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-984z2" event={"ID":"05290855-3013-4ad2-9be1-c2ac618bd75a","Type":"ContainerStarted","Data":"54753175a15b0bbc20be98dcebb6052f3d8232e28506859b5a5210cb8a145c05"} Jul 08 19:32:03.292844 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:03.292820 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" event={"ID":"217f7efe-cd01-4b57-8e98-3d52d6420125","Type":"ContainerStarted","Data":"6126da60cc1d9f9ecef6e4622576adc7c431cf12e4de3ca294bc5b68eb4f3961"} Jul 08 19:32:03.308719 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:03.308666 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-984z2" podStartSLOduration=2.377391358 podStartE2EDuration="5.308650209s" totalImagesPullingTime="2.931258851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:31:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:03.307567309 +0000 UTC m=+484.965005945" watchObservedRunningTime="2026-07-08 19:32:03.308650209 +0000 UTC m=+484.966088815" Jul 08 19:32:06.307269 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:06.307233 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" event={"ID":"217f7efe-cd01-4b57-8e98-3d52d6420125","Type":"ContainerStarted","Data":"ecb5d9d53cf027999b5b648d4179ca7a621eb5623244aa3cadff1140499bac17"} Jul 08 19:32:06.325182 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:06.325133 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sgr4q" podStartSLOduration=1.92743385 podStartE2EDuration="4.325114984s" totalImagesPullingTime="2.397681134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:06.323888813 +0000 UTC m=+487.981327420" watchObservedRunningTime="2026-07-08 19:32:06.325114984 +0000 UTC m=+487.982553590" Jul 08 19:32:08.880982 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.880942 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr"] Jul 08 19:32:08.883227 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.883211 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:08.885663 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.885642 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 19:32:08.886624 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.886605 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2jsr5\"" Jul 08 19:32:08.886686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.886605 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 19:32:08.893096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.893070 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr"] Jul 08 19:32:08.898786 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.898759 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ntdl\" (UniqueName: \"kubernetes.io/projected/d68f7aaf-8823-4c35-924b-bba97cb74aa1-kube-api-access-9ntdl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:08.898906 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.898818 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:08.898906 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.898878 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:08.999714 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.999672 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:08.999714 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.999714 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:08.999915 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:08.999776 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ntdl\" (UniqueName: \"kubernetes.io/projected/d68f7aaf-8823-4c35-924b-bba97cb74aa1-kube-api-access-9ntdl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:09.000092 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:09.000072 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:09.000133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:09.000107 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:09.008222 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:09.008199 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ntdl\" (UniqueName: \"kubernetes.io/projected/d68f7aaf-8823-4c35-924b-bba97cb74aa1-kube-api-access-9ntdl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:09.192243 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:09.192156 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:09.320157 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:09.320128 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr"] Jul 08 19:32:09.322915 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:09.322886 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd68f7aaf_8823_4c35_924b_bba97cb74aa1.slice/crio-0b34c4c64b4abf044b0b3f46ea2225eb8ec72bb05afa399b7615b849339a4158 WatchSource:0}: Error finding container 0b34c4c64b4abf044b0b3f46ea2225eb8ec72bb05afa399b7615b849339a4158: Status 404 returned error can't find the container with id 0b34c4c64b4abf044b0b3f46ea2225eb8ec72bb05afa399b7615b849339a4158 Jul 08 19:32:10.324717 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:10.324622 2573 generic.go:356] "Generic (PLEG): container finished" podID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerID="ae3da6e32e857e09c1e080380a4a7596c666a569f1a1e08a794dbce49ab75bc9" exitCode=0 Jul 08 19:32:10.325117 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:10.324712 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" event={"ID":"d68f7aaf-8823-4c35-924b-bba97cb74aa1","Type":"ContainerDied","Data":"ae3da6e32e857e09c1e080380a4a7596c666a569f1a1e08a794dbce49ab75bc9"} Jul 08 19:32:10.325117 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:10.324772 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" event={"ID":"d68f7aaf-8823-4c35-924b-bba97cb74aa1","Type":"ContainerStarted","Data":"0b34c4c64b4abf044b0b3f46ea2225eb8ec72bb05afa399b7615b849339a4158"} Jul 08 19:32:11.330419 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:11.330332 2573 generic.go:356] "Generic (PLEG): container finished" podID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerID="4cc992fd0edf5b37f4f69f93e65a7d5a54f4a68c346d730432845cc26e79a736" exitCode=0 Jul 08 19:32:11.330807 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:11.330413 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" event={"ID":"d68f7aaf-8823-4c35-924b-bba97cb74aa1","Type":"ContainerDied","Data":"4cc992fd0edf5b37f4f69f93e65a7d5a54f4a68c346d730432845cc26e79a736"} Jul 08 19:32:12.336927 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:12.336894 2573 generic.go:356] "Generic (PLEG): container finished" podID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerID="4451ae2dd8f184d9445d5758fa4adc6d913a82cf564d5974cd5beed6afd6d033" exitCode=0 Jul 08 19:32:12.337321 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:12.336963 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" event={"ID":"d68f7aaf-8823-4c35-924b-bba97cb74aa1","Type":"ContainerDied","Data":"4451ae2dd8f184d9445d5758fa4adc6d913a82cf564d5974cd5beed6afd6d033"} Jul 08 19:32:13.462848 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.462820 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:13.533264 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.533231 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ntdl\" (UniqueName: \"kubernetes.io/projected/d68f7aaf-8823-4c35-924b-bba97cb74aa1-kube-api-access-9ntdl\") pod \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " Jul 08 19:32:13.533412 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.533318 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-bundle\") pod \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " Jul 08 19:32:13.533412 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.533376 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-util\") pod \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\" (UID: \"d68f7aaf-8823-4c35-924b-bba97cb74aa1\") " Jul 08 19:32:13.534013 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.533988 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-bundle" (OuterVolumeSpecName: "bundle") pod "d68f7aaf-8823-4c35-924b-bba97cb74aa1" (UID: "d68f7aaf-8823-4c35-924b-bba97cb74aa1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:13.535375 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.535353 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d68f7aaf-8823-4c35-924b-bba97cb74aa1-kube-api-access-9ntdl" (OuterVolumeSpecName: "kube-api-access-9ntdl") pod "d68f7aaf-8823-4c35-924b-bba97cb74aa1" (UID: "d68f7aaf-8823-4c35-924b-bba97cb74aa1"). InnerVolumeSpecName "kube-api-access-9ntdl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:32:13.538580 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.538545 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-util" (OuterVolumeSpecName: "util") pod "d68f7aaf-8823-4c35-924b-bba97cb74aa1" (UID: "d68f7aaf-8823-4c35-924b-bba97cb74aa1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:13.634079 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.633971 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-util\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:13.634079 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.633999 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ntdl\" (UniqueName: \"kubernetes.io/projected/d68f7aaf-8823-4c35-924b-bba97cb74aa1-kube-api-access-9ntdl\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:13.634079 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:13.634010 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d68f7aaf-8823-4c35-924b-bba97cb74aa1-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:14.346589 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:14.346562 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" Jul 08 19:32:14.346589 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:14.346582 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqzlr" event={"ID":"d68f7aaf-8823-4c35-924b-bba97cb74aa1","Type":"ContainerDied","Data":"0b34c4c64b4abf044b0b3f46ea2225eb8ec72bb05afa399b7615b849339a4158"} Jul 08 19:32:14.346803 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:14.346624 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b34c4c64b4abf044b0b3f46ea2225eb8ec72bb05afa399b7615b849339a4158" Jul 08 19:32:25.369666 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.369631 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk"] Jul 08 19:32:25.370096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.369878 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerName="pull" Jul 08 19:32:25.370096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.369889 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerName="pull" Jul 08 19:32:25.370096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.369905 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerName="util" Jul 08 19:32:25.370096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.369911 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerName="util" Jul 08 19:32:25.370096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.369923 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerName="extract" Jul 08 19:32:25.370096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.369928 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerName="extract" Jul 08 19:32:25.370096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.369983 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d68f7aaf-8823-4c35-924b-bba97cb74aa1" containerName="extract" Jul 08 19:32:25.371913 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.371895 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.374650 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.374631 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 19:32:25.374762 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.374665 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g9lls\"" Jul 08 19:32:25.374762 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.374682 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 19:32:25.374762 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.374699 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 19:32:25.374888 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.374811 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 19:32:25.390392 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.390361 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk"] Jul 08 19:32:25.417829 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.417797 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxcq5\" (UniqueName: \"kubernetes.io/projected/ea1c2768-c0e8-4e42-89a1-37f2298434e6-kube-api-access-wxcq5\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.417988 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.417848 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ea1c2768-c0e8-4e42-89a1-37f2298434e6-manifests\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.417988 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.417919 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ea1c2768-c0e8-4e42-89a1-37f2298434e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.417988 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.417980 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea1c2768-c0e8-4e42-89a1-37f2298434e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.418114 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.418002 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea1c2768-c0e8-4e42-89a1-37f2298434e6-tmp\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.518836 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.518794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ea1c2768-c0e8-4e42-89a1-37f2298434e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.519049 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.518854 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea1c2768-c0e8-4e42-89a1-37f2298434e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.519049 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.518874 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea1c2768-c0e8-4e42-89a1-37f2298434e6-tmp\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.519049 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.518893 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxcq5\" (UniqueName: \"kubernetes.io/projected/ea1c2768-c0e8-4e42-89a1-37f2298434e6-kube-api-access-wxcq5\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.519049 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.518921 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ea1c2768-c0e8-4e42-89a1-37f2298434e6-manifests\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.519388 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.519362 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ea1c2768-c0e8-4e42-89a1-37f2298434e6-manifests\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.521279 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.521249 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ea1c2768-c0e8-4e42-89a1-37f2298434e6-tmp\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.521360 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.521310 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ea1c2768-c0e8-4e42-89a1-37f2298434e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.521408 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.521396 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ea1c2768-c0e8-4e42-89a1-37f2298434e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.527282 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.527259 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxcq5\" (UniqueName: \"kubernetes.io/projected/ea1c2768-c0e8-4e42-89a1-37f2298434e6-kube-api-access-wxcq5\") pod \"opendatahub-operator-controller-manager-77ddc6fd87-kvrlk\" (UID: \"ea1c2768-c0e8-4e42-89a1-37f2298434e6\") " pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.687682 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.687586 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:25.821852 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:25.821817 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk"] Jul 08 19:32:25.827472 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:25.827445 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1c2768_c0e8_4e42_89a1_37f2298434e6.slice/crio-f673c3445595bc8e252bc8931197b8eabe0626f94e756ae0bd335a21af5c8678 WatchSource:0}: Error finding container f673c3445595bc8e252bc8931197b8eabe0626f94e756ae0bd335a21af5c8678: Status 404 returned error can't find the container with id f673c3445595bc8e252bc8931197b8eabe0626f94e756ae0bd335a21af5c8678 Jul 08 19:32:26.396490 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:26.396457 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" event={"ID":"ea1c2768-c0e8-4e42-89a1-37f2298434e6","Type":"ContainerStarted","Data":"f673c3445595bc8e252bc8931197b8eabe0626f94e756ae0bd335a21af5c8678"} Jul 08 19:32:29.414584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:29.414543 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" event={"ID":"ea1c2768-c0e8-4e42-89a1-37f2298434e6","Type":"ContainerStarted","Data":"69156845e3b9cc08ea654a328a465afe419425803d7c1582a61163be7a0c868e"} Jul 08 19:32:30.420630 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:30.420538 2573 generic.go:356] "Generic (PLEG): container finished" podID="ea1c2768-c0e8-4e42-89a1-37f2298434e6" containerID="69156845e3b9cc08ea654a328a465afe419425803d7c1582a61163be7a0c868e" exitCode=0 Jul 08 19:32:30.420630 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:30.420585 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" event={"ID":"ea1c2768-c0e8-4e42-89a1-37f2298434e6","Type":"ContainerDied","Data":"69156845e3b9cc08ea654a328a465afe419425803d7c1582a61163be7a0c868e"} Jul 08 19:32:31.427108 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:31.427069 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" event={"ID":"ea1c2768-c0e8-4e42-89a1-37f2298434e6","Type":"ContainerStarted","Data":"6920026d489ded06361f007fec7cccf8dca3866b935fbcc7360c9da3078a5e2a"} Jul 08 19:32:31.427484 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:31.427227 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:31.450181 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:31.450118 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" podStartSLOduration=1.895066876 podStartE2EDuration="6.450098121s" totalImagesPullingTime="3.547640697s" totalInitContainerRuntime="1.007390548s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:31.449208215 +0000 UTC m=+513.106646844" watchObservedRunningTime="2026-07-08 19:32:31.450098121 +0000 UTC m=+513.107536728" Jul 08 19:32:42.433939 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:42.433907 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-77ddc6fd87-kvrlk" Jul 08 19:32:45.501374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.501338 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b84bffb4-djnqq"] Jul 08 19:32:45.509838 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.509808 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.512136 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.512110 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 19:32:45.513156 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.513139 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 19:32:45.513281 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.513155 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9gk5k\"" Jul 08 19:32:45.513281 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.513167 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 19:32:45.513281 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.513160 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 19:32:45.515731 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.515711 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b84bffb4-djnqq"] Jul 08 19:32:45.586943 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.586902 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5674a395-0063-49c9-8316-3c2302623d8b-tls-certs\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.586943 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.586943 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4t8k\" (UniqueName: \"kubernetes.io/projected/5674a395-0063-49c9-8316-3c2302623d8b-kube-api-access-f4t8k\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.587174 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.586989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5674a395-0063-49c9-8316-3c2302623d8b-tmp\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.673914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.673872 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nlj55"] Jul 08 19:32:45.677665 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.677646 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:45.681087 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.681056 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 19:32:45.681202 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.681058 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-bb7vm\"" Jul 08 19:32:45.685700 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.685613 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nlj55"] Jul 08 19:32:45.687538 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.687509 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5674a395-0063-49c9-8316-3c2302623d8b-tls-certs\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.687645 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.687556 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4t8k\" (UniqueName: \"kubernetes.io/projected/5674a395-0063-49c9-8316-3c2302623d8b-kube-api-access-f4t8k\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.687645 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.687614 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5674a395-0063-49c9-8316-3c2302623d8b-tmp\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.689955 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.689924 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5674a395-0063-49c9-8316-3c2302623d8b-tmp\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.690308 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.690288 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5674a395-0063-49c9-8316-3c2302623d8b-tls-certs\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.703684 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.703652 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4t8k\" (UniqueName: \"kubernetes.io/projected/5674a395-0063-49c9-8316-3c2302623d8b-kube-api-access-f4t8k\") pod \"kube-auth-proxy-b84bffb4-djnqq\" (UID: \"5674a395-0063-49c9-8316-3c2302623d8b\") " pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.788275 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.788187 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slmrn\" (UniqueName: \"kubernetes.io/projected/f241326d-2cef-4908-9037-c58572d8ff09-kube-api-access-slmrn\") pod \"odh-model-controller-7998c67dc4-nlj55\" (UID: \"f241326d-2cef-4908-9037-c58572d8ff09\") " pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:45.788275 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.788233 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f241326d-2cef-4908-9037-c58572d8ff09-cert\") pod \"odh-model-controller-7998c67dc4-nlj55\" (UID: \"f241326d-2cef-4908-9037-c58572d8ff09\") " pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:45.820308 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.820271 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" Jul 08 19:32:45.888794 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.888752 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slmrn\" (UniqueName: \"kubernetes.io/projected/f241326d-2cef-4908-9037-c58572d8ff09-kube-api-access-slmrn\") pod \"odh-model-controller-7998c67dc4-nlj55\" (UID: \"f241326d-2cef-4908-9037-c58572d8ff09\") " pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:45.888927 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.888820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f241326d-2cef-4908-9037-c58572d8ff09-cert\") pod \"odh-model-controller-7998c67dc4-nlj55\" (UID: \"f241326d-2cef-4908-9037-c58572d8ff09\") " pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:45.892165 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.892143 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f241326d-2cef-4908-9037-c58572d8ff09-cert\") pod \"odh-model-controller-7998c67dc4-nlj55\" (UID: \"f241326d-2cef-4908-9037-c58572d8ff09\") " pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:45.909824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.909780 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slmrn\" (UniqueName: \"kubernetes.io/projected/f241326d-2cef-4908-9037-c58572d8ff09-kube-api-access-slmrn\") pod \"odh-model-controller-7998c67dc4-nlj55\" (UID: \"f241326d-2cef-4908-9037-c58572d8ff09\") " pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:45.981970 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.981940 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b84bffb4-djnqq"] Jul 08 19:32:45.985381 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:45.985343 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5674a395_0063_49c9_8316_3c2302623d8b.slice/crio-98b56bad945b9bc7dc9356caae94c77246cc74eaad1664bd64cf066391080d1e WatchSource:0}: Error finding container 98b56bad945b9bc7dc9356caae94c77246cc74eaad1664bd64cf066391080d1e: Status 404 returned error can't find the container with id 98b56bad945b9bc7dc9356caae94c77246cc74eaad1664bd64cf066391080d1e Jul 08 19:32:45.989132 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:45.989111 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:46.136874 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.136847 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-nlj55"] Jul 08 19:32:46.139779 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:46.139750 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf241326d_2cef_4908_9037_c58572d8ff09.slice/crio-74b0379b5ef601120719dff531128603bf9ed0be9b4abcbd220ceaca507b7017 WatchSource:0}: Error finding container 74b0379b5ef601120719dff531128603bf9ed0be9b4abcbd220ceaca507b7017: Status 404 returned error can't find the container with id 74b0379b5ef601120719dff531128603bf9ed0be9b4abcbd220ceaca507b7017 Jul 08 19:32:46.495330 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.495294 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" event={"ID":"f241326d-2cef-4908-9037-c58572d8ff09","Type":"ContainerStarted","Data":"74b0379b5ef601120719dff531128603bf9ed0be9b4abcbd220ceaca507b7017"} Jul 08 19:32:46.496309 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.496288 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" event={"ID":"5674a395-0063-49c9-8316-3c2302623d8b","Type":"ContainerStarted","Data":"98b56bad945b9bc7dc9356caae94c77246cc74eaad1664bd64cf066391080d1e"} Jul 08 19:32:46.677217 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.677180 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh"] Jul 08 19:32:46.682064 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.682039 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.684871 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.684847 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 19:32:46.685007 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.684852 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6cs98\"" Jul 08 19:32:46.685241 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.685220 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 08 19:32:46.685356 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.685253 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 08 19:32:46.685356 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.685257 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 08 19:32:46.693608 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.693582 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh"] Jul 08 19:32:46.796664 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.796572 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55668\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-kube-api-access-55668\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.796664 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.796618 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dbb2cd-0467-4dac-b129-2d28e77d309b-local-certs\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.796844 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.796684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.796844 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.796719 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-token\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.796844 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.796740 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-cacerts\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.796844 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.796757 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.796844 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.796824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.897289 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.897250 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dbb2cd-0467-4dac-b129-2d28e77d309b-local-certs\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.897492 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.897316 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.897492 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.897340 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-token\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.897492 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.897357 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-cacerts\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.897492 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.897378 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.897492 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.897415 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.897492 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.897476 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55668\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-kube-api-access-55668\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.898686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.898644 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.899992 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.899965 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dbb2cd-0467-4dac-b129-2d28e77d309b-local-certs\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.900588 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.900531 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.900688 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.900646 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.900747 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.900736 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-cacerts\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.905602 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.905564 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-token\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:46.905764 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:46.905747 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55668\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-kube-api-access-55668\") pod \"istiod-openshift-gateway-7574459656-cjwnh\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:47.005810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:47.005770 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:47.167700 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:47.167465 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh"] Jul 08 19:32:47.170404 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:47.170371 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72dbb2cd_0467_4dac_b129_2d28e77d309b.slice/crio-236fef91e0e7a45a4326faf6c14d6c5f290654b10595e75a27c9edb493bf30c6 WatchSource:0}: Error finding container 236fef91e0e7a45a4326faf6c14d6c5f290654b10595e75a27c9edb493bf30c6: Status 404 returned error can't find the container with id 236fef91e0e7a45a4326faf6c14d6c5f290654b10595e75a27c9edb493bf30c6 Jul 08 19:32:47.502437 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:47.502402 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" event={"ID":"72dbb2cd-0467-4dac-b129-2d28e77d309b","Type":"ContainerStarted","Data":"236fef91e0e7a45a4326faf6c14d6c5f290654b10595e75a27c9edb493bf30c6"} Jul 08 19:32:51.816973 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:51.816930 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:32:51.817581 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:51.817015 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:32:52.532584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.532456 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" event={"ID":"72dbb2cd-0467-4dac-b129-2d28e77d309b","Type":"ContainerStarted","Data":"988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9"} Jul 08 19:32:52.532797 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.532722 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:52.534697 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.534651 2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-cjwnh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 19:32:52.534837 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.534717 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" podUID="72dbb2cd-0467-4dac-b129-2d28e77d309b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:32:52.535190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.534941 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" event={"ID":"f241326d-2cef-4908-9037-c58572d8ff09","Type":"ContainerStarted","Data":"259e86ccb45e9a4712e0e861625b53e53c0ddda35fef40c9e529c2e778edde8b"} Jul 08 19:32:52.535190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.535088 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:32:52.536923 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.536897 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" event={"ID":"5674a395-0063-49c9-8316-3c2302623d8b","Type":"ContainerStarted","Data":"9b0f2e7aec74f2d871ac4e694d0a79c27b4a616c87fa6d3503fadd39daf53a57"} Jul 08 19:32:52.553985 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.553914 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" podStartSLOduration=1.910061544 podStartE2EDuration="6.553892369s" totalImagesPullingTime="4.643830825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:52.552398971 +0000 UTC m=+534.209837578" watchObservedRunningTime="2026-07-08 19:32:52.553892369 +0000 UTC m=+534.211330972" Jul 08 19:32:52.570204 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.570141 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" podStartSLOduration=1.889200194 podStartE2EDuration="7.57012096s" totalImagesPullingTime="5.680920766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:52.567786712 +0000 UTC m=+534.225225318" watchObservedRunningTime="2026-07-08 19:32:52.57012096 +0000 UTC m=+534.227559567" Jul 08 19:32:52.585148 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:52.585090 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b84bffb4-djnqq" podStartSLOduration=1.757292292 podStartE2EDuration="7.585072942s" totalImagesPullingTime="5.82778065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:52.583310573 +0000 UTC m=+534.240749180" watchObservedRunningTime="2026-07-08 19:32:52.585072942 +0000 UTC m=+534.242511548" Jul 08 19:32:53.544427 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:53.544388 2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-cjwnh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 19:32:53.544889 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:53.544457 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" podUID="72dbb2cd-0467-4dac-b129-2d28e77d309b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 19:32:54.312758 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.312717 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp"] Jul 08 19:32:54.315872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.315852 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.318216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.318195 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xbvv4\"" Jul 08 19:32:54.327072 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.327044 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp"] Jul 08 19:32:54.473625 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473588 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.473791 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473634 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/551885ec-5d6d-431e-af1c-7318fcf75bcc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.473791 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473666 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.473791 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473687 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.473791 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473710 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.473791 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473788 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.473966 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473812 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.473966 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473835 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.473966 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.473857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx946\" (UniqueName: \"kubernetes.io/projected/551885ec-5d6d-431e-af1c-7318fcf75bcc-kube-api-access-bx946\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.574661 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574578 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/551885ec-5d6d-431e-af1c-7318fcf75bcc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.574661 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574615 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.574661 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574634 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.574661 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574659 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574721 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574756 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574787 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx946\" (UniqueName: \"kubernetes.io/projected/551885ec-5d6d-431e-af1c-7318fcf75bcc-kube-api-access-bx946\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.574850 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.575054 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.575223 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.575284 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.575291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.575712 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.575688 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/551885ec-5d6d-431e-af1c-7318fcf75bcc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.577119 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.577094 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.577721 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.577683 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.590002 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.589976 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/551885ec-5d6d-431e-af1c-7318fcf75bcc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.590948 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.590924 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx946\" (UniqueName: \"kubernetes.io/projected/551885ec-5d6d-431e-af1c-7318fcf75bcc-kube-api-access-bx946\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mzqnp\" (UID: \"551885ec-5d6d-431e-af1c-7318fcf75bcc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.626428 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.626376 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:54.769510 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:54.769486 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp"] Jul 08 19:32:54.771365 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:54.771338 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551885ec_5d6d_431e_af1c_7318fcf75bcc.slice/crio-6ab1f88a0e2f68964ab41bbfdc0162078d76c74a3277de155de3d8cd899ddaa4 WatchSource:0}: Error finding container 6ab1f88a0e2f68964ab41bbfdc0162078d76c74a3277de155de3d8cd899ddaa4: Status 404 returned error can't find the container with id 6ab1f88a0e2f68964ab41bbfdc0162078d76c74a3277de155de3d8cd899ddaa4 Jul 08 19:32:55.554737 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:55.554696 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" event={"ID":"551885ec-5d6d-431e-af1c-7318fcf75bcc","Type":"ContainerStarted","Data":"6ab1f88a0e2f68964ab41bbfdc0162078d76c74a3277de155de3d8cd899ddaa4"} Jul 08 19:32:56.545475 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.545446 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:56.798050 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.797941 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr"] Jul 08 19:32:56.804816 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.804794 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:32:56.807400 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.807380 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 08 19:32:56.808116 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.808097 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6hcxk\"" Jul 08 19:32:56.814280 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.814261 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6"] Jul 08 19:32:56.817884 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.817863 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr"] Jul 08 19:32:56.818006 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.817984 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:32:56.820317 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.820296 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 08 19:32:56.820516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.820303 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vkjqn\"" Jul 08 19:32:56.836982 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.836952 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6"] Jul 08 19:32:56.996354 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.996315 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c71dc895-238f-48a9-9b32-b672b43ba401-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9d6v6\" (UID: \"c71dc895-238f-48a9-9b32-b672b43ba401\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:32:56.996544 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.996365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwksc\" (UniqueName: \"kubernetes.io/projected/c71dc895-238f-48a9-9b32-b672b43ba401-kube-api-access-wwksc\") pod \"llmisvc-controller-manager-777c6bd7d5-9d6v6\" (UID: \"c71dc895-238f-48a9-9b32-b672b43ba401\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:32:56.996625 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.996552 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39a574bc-8799-4a4a-b869-21843ab70b28-cert\") pod \"kserve-controller-manager-d6f4b65b8-bkcpr\" (UID: \"39a574bc-8799-4a4a-b869-21843ab70b28\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:32:56.996625 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:56.996592 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47mxd\" (UniqueName: \"kubernetes.io/projected/39a574bc-8799-4a4a-b869-21843ab70b28-kube-api-access-47mxd\") pod \"kserve-controller-manager-d6f4b65b8-bkcpr\" (UID: \"39a574bc-8799-4a4a-b869-21843ab70b28\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:32:57.097743 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.097666 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39a574bc-8799-4a4a-b869-21843ab70b28-cert\") pod \"kserve-controller-manager-d6f4b65b8-bkcpr\" (UID: \"39a574bc-8799-4a4a-b869-21843ab70b28\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:32:57.097743 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.097706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47mxd\" (UniqueName: \"kubernetes.io/projected/39a574bc-8799-4a4a-b869-21843ab70b28-kube-api-access-47mxd\") pod \"kserve-controller-manager-d6f4b65b8-bkcpr\" (UID: \"39a574bc-8799-4a4a-b869-21843ab70b28\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:32:57.097743 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.097742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c71dc895-238f-48a9-9b32-b672b43ba401-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9d6v6\" (UID: \"c71dc895-238f-48a9-9b32-b672b43ba401\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:32:57.098053 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.097760 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwksc\" (UniqueName: \"kubernetes.io/projected/c71dc895-238f-48a9-9b32-b672b43ba401-kube-api-access-wwksc\") pod \"llmisvc-controller-manager-777c6bd7d5-9d6v6\" (UID: \"c71dc895-238f-48a9-9b32-b672b43ba401\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:32:57.100444 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.100417 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/39a574bc-8799-4a4a-b869-21843ab70b28-cert\") pod \"kserve-controller-manager-d6f4b65b8-bkcpr\" (UID: \"39a574bc-8799-4a4a-b869-21843ab70b28\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:32:57.100561 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.100506 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c71dc895-238f-48a9-9b32-b672b43ba401-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9d6v6\" (UID: \"c71dc895-238f-48a9-9b32-b672b43ba401\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:32:57.106245 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.106223 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47mxd\" (UniqueName: \"kubernetes.io/projected/39a574bc-8799-4a4a-b869-21843ab70b28-kube-api-access-47mxd\") pod \"kserve-controller-manager-d6f4b65b8-bkcpr\" (UID: \"39a574bc-8799-4a4a-b869-21843ab70b28\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:32:57.106487 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.106464 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwksc\" (UniqueName: \"kubernetes.io/projected/c71dc895-238f-48a9-9b32-b672b43ba401-kube-api-access-wwksc\") pod \"llmisvc-controller-manager-777c6bd7d5-9d6v6\" (UID: \"c71dc895-238f-48a9-9b32-b672b43ba401\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:32:57.117340 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.117304 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:32:57.131610 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.131582 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:32:57.450355 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.450246 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:32:57.450355 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.450322 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:32:57.450355 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.450349 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:32:57.537772 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.537742 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr"] Jul 08 19:32:57.541490 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:57.541463 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a574bc_8799_4a4a_b869_21843ab70b28.slice/crio-2b6c35bdb0694d25cb793f1efb67e78536db703492882fb7f0151188612e8548 WatchSource:0}: Error finding container 2b6c35bdb0694d25cb793f1efb67e78536db703492882fb7f0151188612e8548: Status 404 returned error can't find the container with id 2b6c35bdb0694d25cb793f1efb67e78536db703492882fb7f0151188612e8548 Jul 08 19:32:57.549615 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.549590 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6"] Jul 08 19:32:57.551184 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:32:57.551162 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc71dc895_238f_48a9_9b32_b672b43ba401.slice/crio-6d695554c2ee7d24386a67808a9bfa21874a8d358a164d37738f207ce7852aa3 WatchSource:0}: Error finding container 6d695554c2ee7d24386a67808a9bfa21874a8d358a164d37738f207ce7852aa3: Status 404 returned error can't find the container with id 6d695554c2ee7d24386a67808a9bfa21874a8d358a164d37738f207ce7852aa3 Jul 08 19:32:57.565726 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.565693 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" event={"ID":"551885ec-5d6d-431e-af1c-7318fcf75bcc","Type":"ContainerStarted","Data":"fdef460da451ca085201f442ff5eb5c8842513f705338f3b5313ee8b8afc9ff6"} Jul 08 19:32:57.567740 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.567705 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" event={"ID":"c71dc895-238f-48a9-9b32-b672b43ba401","Type":"ContainerStarted","Data":"6d695554c2ee7d24386a67808a9bfa21874a8d358a164d37738f207ce7852aa3"} Jul 08 19:32:57.569051 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.569009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" event={"ID":"39a574bc-8799-4a4a-b869-21843ab70b28","Type":"ContainerStarted","Data":"2b6c35bdb0694d25cb793f1efb67e78536db703492882fb7f0151188612e8548"} Jul 08 19:32:57.614849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.614800 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" podStartSLOduration=0.938126526 podStartE2EDuration="3.614783554s" totalImagesPullingTime="2.676657028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:32:57.613214771 +0000 UTC m=+539.270653416" watchObservedRunningTime="2026-07-08 19:32:57.614783554 +0000 UTC m=+539.272222159" Jul 08 19:32:57.627458 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:57.627412 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:58.600146 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:58.600108 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh"] Jul 08 19:32:58.600534 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:58.600334 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" podUID="72dbb2cd-0467-4dac-b129-2d28e77d309b" containerName="discovery" containerID="cri-o://988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9" gracePeriod=30 Jul 08 19:32:58.633185 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:58.632324 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:58.957670 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:58.956633 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:59.115784 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.115745 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-kubeconfig\") pod \"72dbb2cd-0467-4dac-b129-2d28e77d309b\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " Jul 08 19:32:59.115952 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.115793 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-dns-cert\") pod \"72dbb2cd-0467-4dac-b129-2d28e77d309b\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " Jul 08 19:32:59.115952 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.115826 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-token\") pod \"72dbb2cd-0467-4dac-b129-2d28e77d309b\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " Jul 08 19:32:59.115952 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.115916 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55668\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-kube-api-access-55668\") pod \"72dbb2cd-0467-4dac-b129-2d28e77d309b\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " Jul 08 19:32:59.115952 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.115952 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-cacerts\") pod \"72dbb2cd-0467-4dac-b129-2d28e77d309b\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " Jul 08 19:32:59.116199 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.115982 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dbb2cd-0467-4dac-b129-2d28e77d309b-local-certs\") pod \"72dbb2cd-0467-4dac-b129-2d28e77d309b\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " Jul 08 19:32:59.116199 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.116009 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-ca-configmap\") pod \"72dbb2cd-0467-4dac-b129-2d28e77d309b\" (UID: \"72dbb2cd-0467-4dac-b129-2d28e77d309b\") " Jul 08 19:32:59.116636 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.116586 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "72dbb2cd-0467-4dac-b129-2d28e77d309b" (UID: "72dbb2cd-0467-4dac-b129-2d28e77d309b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:32:59.120052 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.120000 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-kube-api-access-55668" (OuterVolumeSpecName: "kube-api-access-55668") pod "72dbb2cd-0467-4dac-b129-2d28e77d309b" (UID: "72dbb2cd-0467-4dac-b129-2d28e77d309b"). InnerVolumeSpecName "kube-api-access-55668". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:32:59.121913 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.121797 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "72dbb2cd-0467-4dac-b129-2d28e77d309b" (UID: "72dbb2cd-0467-4dac-b129-2d28e77d309b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:32:59.121913 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.121815 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-token" (OuterVolumeSpecName: "istio-token") pod "72dbb2cd-0467-4dac-b129-2d28e77d309b" (UID: "72dbb2cd-0467-4dac-b129-2d28e77d309b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:32:59.121913 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.121884 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72dbb2cd-0467-4dac-b129-2d28e77d309b-local-certs" (OuterVolumeSpecName: "local-certs") pod "72dbb2cd-0467-4dac-b129-2d28e77d309b" (UID: "72dbb2cd-0467-4dac-b129-2d28e77d309b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:32:59.122620 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.122585 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "72dbb2cd-0467-4dac-b129-2d28e77d309b" (UID: "72dbb2cd-0467-4dac-b129-2d28e77d309b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:32:59.123125 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.123087 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-cacerts" (OuterVolumeSpecName: "cacerts") pod "72dbb2cd-0467-4dac-b129-2d28e77d309b" (UID: "72dbb2cd-0467-4dac-b129-2d28e77d309b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:32:59.217218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.217130 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55668\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-kube-api-access-55668\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:59.217218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.217162 2573 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-cacerts\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:59.217218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.217174 2573 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dbb2cd-0467-4dac-b129-2d28e77d309b-local-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:59.217218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.217183 2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-ca-configmap\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:59.217218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.217191 2573 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-kubeconfig\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:59.217218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.217199 2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-csr-dns-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:59.217218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.217208 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dbb2cd-0467-4dac-b129-2d28e77d309b-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:32:59.581439 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.581404 2573 generic.go:356] "Generic (PLEG): container finished" podID="72dbb2cd-0467-4dac-b129-2d28e77d309b" containerID="988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9" exitCode=0 Jul 08 19:32:59.581619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.581463 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" Jul 08 19:32:59.581619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.581483 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" event={"ID":"72dbb2cd-0467-4dac-b129-2d28e77d309b","Type":"ContainerDied","Data":"988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9"} Jul 08 19:32:59.581619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.581535 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh" event={"ID":"72dbb2cd-0467-4dac-b129-2d28e77d309b","Type":"ContainerDied","Data":"236fef91e0e7a45a4326faf6c14d6c5f290654b10595e75a27c9edb493bf30c6"} Jul 08 19:32:59.581619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.581555 2573 scope.go:117] "RemoveContainer" containerID="988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9" Jul 08 19:32:59.581956 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.581937 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:59.582938 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.582912 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mzqnp" Jul 08 19:32:59.591745 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.591726 2573 scope.go:117] "RemoveContainer" containerID="988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9" Jul 08 19:32:59.592085 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:32:59.592061 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9\": container with ID starting with 988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9 not found: ID does not exist" containerID="988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9" Jul 08 19:32:59.592176 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.592096 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9"} err="failed to get container status \"988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9\": rpc error: code = NotFound desc = could not find container \"988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9\": container with ID starting with 988ea982e4e89057eb04d30e9ec09ae5a5888a40d9a5895e55cf87b1bb8814b9 not found: ID does not exist" Jul 08 19:32:59.651451 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.651413 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh"] Jul 08 19:32:59.654456 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:32:59.654430 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cjwnh"] Jul 08 19:33:00.568584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:00.568439 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72dbb2cd-0467-4dac-b129-2d28e77d309b" path="/var/lib/kubelet/pods/72dbb2cd-0467-4dac-b129-2d28e77d309b/volumes" Jul 08 19:33:01.594080 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:01.594043 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" event={"ID":"c71dc895-238f-48a9-9b32-b672b43ba401","Type":"ContainerStarted","Data":"1323d9ef0cc217e3c5dbf03b8ef5a576b48307f8beee6ff9d89ec644aba7620b"} Jul 08 19:33:01.594544 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:01.594105 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:33:01.595688 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:01.595666 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" event={"ID":"39a574bc-8799-4a4a-b869-21843ab70b28","Type":"ContainerStarted","Data":"4bf25d565ff9c97c4dd73ec58407576a498364c5a1a2a9639920763736feb5b1"} Jul 08 19:33:01.595887 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:01.595872 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:33:01.616467 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:01.616418 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" podStartSLOduration=1.6796631469999999 podStartE2EDuration="5.61640259s" totalImagesPullingTime="3.936739443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:33:01.614808201 +0000 UTC m=+543.272246809" watchObservedRunningTime="2026-07-08 19:33:01.61640259 +0000 UTC m=+543.273841195" Jul 08 19:33:01.634135 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:01.634083 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" podStartSLOduration=1.691208394 podStartE2EDuration="5.634064026s" totalImagesPullingTime="3.942855632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:33:01.632747684 +0000 UTC m=+543.290186314" watchObservedRunningTime="2026-07-08 19:33:01.634064026 +0000 UTC m=+543.291502632" Jul 08 19:33:03.546404 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:03.546367 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-nlj55" Jul 08 19:33:32.603256 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:32.603224 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9d6v6" Jul 08 19:33:32.606207 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:32.606182 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-bkcpr" Jul 08 19:33:46.718550 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.718514 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n8x7t"] Jul 08 19:33:46.718930 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.718837 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72dbb2cd-0467-4dac-b129-2d28e77d309b" containerName="discovery" Jul 08 19:33:46.718930 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.718850 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="72dbb2cd-0467-4dac-b129-2d28e77d309b" containerName="discovery" Jul 08 19:33:46.718930 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.718904 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="72dbb2cd-0467-4dac-b129-2d28e77d309b" containerName="discovery" Jul 08 19:33:46.728969 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.728939 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" Jul 08 19:33:46.731347 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.731323 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 19:33:46.731611 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.731589 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 19:33:46.732306 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.732285 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-w577p\"" Jul 08 19:33:46.736741 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.736325 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n8x7t"] Jul 08 19:33:46.817554 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.817522 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7hvn\" (UniqueName: \"kubernetes.io/projected/8405f6ab-425d-43b1-be27-04387d7e9b8d-kube-api-access-j7hvn\") pod \"kuadrant-operator-catalog-n8x7t\" (UID: \"8405f6ab-425d-43b1-be27-04387d7e9b8d\") " pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" Jul 08 19:33:46.918214 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.918174 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7hvn\" (UniqueName: \"kubernetes.io/projected/8405f6ab-425d-43b1-be27-04387d7e9b8d-kube-api-access-j7hvn\") pod \"kuadrant-operator-catalog-n8x7t\" (UID: \"8405f6ab-425d-43b1-be27-04387d7e9b8d\") " pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" Jul 08 19:33:46.926281 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:46.926251 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7hvn\" (UniqueName: \"kubernetes.io/projected/8405f6ab-425d-43b1-be27-04387d7e9b8d-kube-api-access-j7hvn\") pod \"kuadrant-operator-catalog-n8x7t\" (UID: \"8405f6ab-425d-43b1-be27-04387d7e9b8d\") " pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" Jul 08 19:33:47.039462 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.039424 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" Jul 08 19:33:47.087585 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.087546 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n8x7t"] Jul 08 19:33:47.178647 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.178621 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n8x7t"] Jul 08 19:33:47.181010 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:33:47.180978 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8405f6ab_425d_43b1_be27_04387d7e9b8d.slice/crio-339d67841330b7b9ad446895290c7495a373c1b47ccd6b98c0d1c4769ce96b03 WatchSource:0}: Error finding container 339d67841330b7b9ad446895290c7495a373c1b47ccd6b98c0d1c4769ce96b03: Status 404 returned error can't find the container with id 339d67841330b7b9ad446895290c7495a373c1b47ccd6b98c0d1c4769ce96b03 Jul 08 19:33:47.287812 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.287778 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2x66b"] Jul 08 19:33:47.290540 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.290497 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:33:47.299790 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.299764 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2x66b"] Jul 08 19:33:47.423393 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.423349 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ghcs\" (UniqueName: \"kubernetes.io/projected/c16e45b4-71de-40a7-8338-201c01495f66-kube-api-access-8ghcs\") pod \"kuadrant-operator-catalog-2x66b\" (UID: \"c16e45b4-71de-40a7-8338-201c01495f66\") " pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:33:47.523965 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.523932 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ghcs\" (UniqueName: \"kubernetes.io/projected/c16e45b4-71de-40a7-8338-201c01495f66-kube-api-access-8ghcs\") pod \"kuadrant-operator-catalog-2x66b\" (UID: \"c16e45b4-71de-40a7-8338-201c01495f66\") " pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:33:47.533395 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.533371 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ghcs\" (UniqueName: \"kubernetes.io/projected/c16e45b4-71de-40a7-8338-201c01495f66-kube-api-access-8ghcs\") pod \"kuadrant-operator-catalog-2x66b\" (UID: \"c16e45b4-71de-40a7-8338-201c01495f66\") " pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:33:47.599691 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.599603 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:33:47.762385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.762349 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2x66b"] Jul 08 19:33:47.789485 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:33:47.789442 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc16e45b4_71de_40a7_8338_201c01495f66.slice/crio-b5114f47c1de307ad642fc1091735f9066d49e0b924ceedd53991027ba8922a5 WatchSource:0}: Error finding container b5114f47c1de307ad642fc1091735f9066d49e0b924ceedd53991027ba8922a5: Status 404 returned error can't find the container with id b5114f47c1de307ad642fc1091735f9066d49e0b924ceedd53991027ba8922a5 Jul 08 19:33:47.802458 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.802430 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" event={"ID":"c16e45b4-71de-40a7-8338-201c01495f66","Type":"ContainerStarted","Data":"b5114f47c1de307ad642fc1091735f9066d49e0b924ceedd53991027ba8922a5"} Jul 08 19:33:47.803677 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:47.803644 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" event={"ID":"8405f6ab-425d-43b1-be27-04387d7e9b8d","Type":"ContainerStarted","Data":"339d67841330b7b9ad446895290c7495a373c1b47ccd6b98c0d1c4769ce96b03"} Jul 08 19:33:49.816415 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:49.816376 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" event={"ID":"c16e45b4-71de-40a7-8338-201c01495f66","Type":"ContainerStarted","Data":"cf047e1dc0f73d018193422563740e333ec8138a326a47aaf6b1439994eb0133"} Jul 08 19:33:49.817775 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:49.817746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" event={"ID":"8405f6ab-425d-43b1-be27-04387d7e9b8d","Type":"ContainerStarted","Data":"8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa"} Jul 08 19:33:49.817886 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:49.817822 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" podUID="8405f6ab-425d-43b1-be27-04387d7e9b8d" containerName="registry-server" containerID="cri-o://8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa" gracePeriod=2 Jul 08 19:33:49.831843 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:49.831799 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" podStartSLOduration=1.341075477 podStartE2EDuration="2.831783434s" totalImagesPullingTime="1.490707957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:33:49.830526398 +0000 UTC m=+591.487965029" watchObservedRunningTime="2026-07-08 19:33:49.831783434 +0000 UTC m=+591.489222017" Jul 08 19:33:49.846134 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:49.846087 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" podStartSLOduration=1.75059347 podStartE2EDuration="3.846072327s" totalImagesPullingTime="2.095478857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:33:49.845107451 +0000 UTC m=+591.502546077" watchObservedRunningTime="2026-07-08 19:33:49.846072327 +0000 UTC m=+591.503510932" Jul 08 19:33:50.058866 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.058839 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" Jul 08 19:33:50.249252 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.249202 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7hvn\" (UniqueName: \"kubernetes.io/projected/8405f6ab-425d-43b1-be27-04387d7e9b8d-kube-api-access-j7hvn\") pod \"8405f6ab-425d-43b1-be27-04387d7e9b8d\" (UID: \"8405f6ab-425d-43b1-be27-04387d7e9b8d\") " Jul 08 19:33:50.251380 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.251349 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8405f6ab-425d-43b1-be27-04387d7e9b8d-kube-api-access-j7hvn" (OuterVolumeSpecName: "kube-api-access-j7hvn") pod "8405f6ab-425d-43b1-be27-04387d7e9b8d" (UID: "8405f6ab-425d-43b1-be27-04387d7e9b8d"). InnerVolumeSpecName "kube-api-access-j7hvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:33:50.349972 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.349928 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7hvn\" (UniqueName: \"kubernetes.io/projected/8405f6ab-425d-43b1-be27-04387d7e9b8d-kube-api-access-j7hvn\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:33:50.823846 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.823809 2573 generic.go:356] "Generic (PLEG): container finished" podID="8405f6ab-425d-43b1-be27-04387d7e9b8d" containerID="8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa" exitCode=0 Jul 08 19:33:50.824374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.823884 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" Jul 08 19:33:50.824374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.823888 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" event={"ID":"8405f6ab-425d-43b1-be27-04387d7e9b8d","Type":"ContainerDied","Data":"8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa"} Jul 08 19:33:50.824374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.823943 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n8x7t" event={"ID":"8405f6ab-425d-43b1-be27-04387d7e9b8d","Type":"ContainerDied","Data":"339d67841330b7b9ad446895290c7495a373c1b47ccd6b98c0d1c4769ce96b03"} Jul 08 19:33:50.824374 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.823963 2573 scope.go:117] "RemoveContainer" containerID="8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa" Jul 08 19:33:50.832882 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.832858 2573 scope.go:117] "RemoveContainer" containerID="8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa" Jul 08 19:33:50.833156 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:33:50.833134 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa\": container with ID starting with 8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa not found: ID does not exist" containerID="8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa" Jul 08 19:33:50.833223 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.833164 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa"} err="failed to get container status \"8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa\": rpc error: code = NotFound desc = could not find container \"8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa\": container with ID starting with 8ff4231ac435a01386dc9497dd55faa39d024f6fd95449b82807aaced85879aa not found: ID does not exist" Jul 08 19:33:50.839240 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.839212 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n8x7t"] Jul 08 19:33:50.844325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:50.844300 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n8x7t"] Jul 08 19:33:52.566865 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:52.566834 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8405f6ab-425d-43b1-be27-04387d7e9b8d" path="/var/lib/kubelet/pods/8405f6ab-425d-43b1-be27-04387d7e9b8d/volumes" Jul 08 19:33:57.600082 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:57.600046 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:33:57.600492 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:57.600105 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:33:57.622794 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:57.622766 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:33:57.877843 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:33:57.877776 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2x66b" Jul 08 19:34:01.700902 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.700864 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p"] Jul 08 19:34:01.701307 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.701209 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8405f6ab-425d-43b1-be27-04387d7e9b8d" containerName="registry-server" Jul 08 19:34:01.701307 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.701222 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8405f6ab-425d-43b1-be27-04387d7e9b8d" containerName="registry-server" Jul 08 19:34:01.701307 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.701279 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8405f6ab-425d-43b1-be27-04387d7e9b8d" containerName="registry-server" Jul 08 19:34:01.704068 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.704049 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.706480 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.706459 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-78nk4\"" Jul 08 19:34:01.713391 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.713369 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p"] Jul 08 19:34:01.744810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.744775 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9srqk\" (UniqueName: \"kubernetes.io/projected/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-kube-api-access-9srqk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.744987 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.744842 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.744987 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.744915 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.845954 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.845919 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.846168 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.845961 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.846168 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.846103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9srqk\" (UniqueName: \"kubernetes.io/projected/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-kube-api-access-9srqk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.846331 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.846315 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.846399 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.846380 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:01.853895 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:01.853865 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9srqk\" (UniqueName: \"kubernetes.io/projected/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-kube-api-access-9srqk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:02.013772 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.013683 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:02.151624 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.151599 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p"] Jul 08 19:34:02.154429 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:02.154397 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbfe2a9d_2ab8_4df8_b52d_a36fc2616889.slice/crio-da138a2e3a82548f76344b0f12185ce92c8817deed597959e4a3e78e0e9ecd96 WatchSource:0}: Error finding container da138a2e3a82548f76344b0f12185ce92c8817deed597959e4a3e78e0e9ecd96: Status 404 returned error can't find the container with id da138a2e3a82548f76344b0f12185ce92c8817deed597959e4a3e78e0e9ecd96 Jul 08 19:34:02.304457 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.304382 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l"] Jul 08 19:34:02.306935 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.306916 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.316874 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.316838 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l"] Jul 08 19:34:02.348751 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.348712 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.348751 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.348760 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsfpl\" (UniqueName: \"kubernetes.io/projected/b47e78cb-e83e-46a8-b391-d15363b90f9f-kube-api-access-dsfpl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.348976 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.348825 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.449275 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.449236 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.449275 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.449276 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsfpl\" (UniqueName: \"kubernetes.io/projected/b47e78cb-e83e-46a8-b391-d15363b90f9f-kube-api-access-dsfpl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.449480 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.449320 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.449633 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.449613 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.449689 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.449668 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.459356 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.459323 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsfpl\" (UniqueName: \"kubernetes.io/projected/b47e78cb-e83e-46a8-b391-d15363b90f9f-kube-api-access-dsfpl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.628608 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.628525 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:02.757760 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.757734 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l"] Jul 08 19:34:02.759891 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:02.759864 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47e78cb_e83e_46a8_b391_d15363b90f9f.slice/crio-b60e64c3e6299059bd90fdd8fcb7ddf1bc077ab70499754f70fc16773377db6b WatchSource:0}: Error finding container b60e64c3e6299059bd90fdd8fcb7ddf1bc077ab70499754f70fc16773377db6b: Status 404 returned error can't find the container with id b60e64c3e6299059bd90fdd8fcb7ddf1bc077ab70499754f70fc16773377db6b Jul 08 19:34:02.880683 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.880598 2573 generic.go:356] "Generic (PLEG): container finished" podID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerID="46c53de79ff0819384804330e6e939f2eeeac24048c47ccf5f4b637df018cb19" exitCode=0 Jul 08 19:34:02.880683 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.880653 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" event={"ID":"b47e78cb-e83e-46a8-b391-d15363b90f9f","Type":"ContainerDied","Data":"46c53de79ff0819384804330e6e939f2eeeac24048c47ccf5f4b637df018cb19"} Jul 08 19:34:02.880876 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.880700 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" event={"ID":"b47e78cb-e83e-46a8-b391-d15363b90f9f","Type":"ContainerStarted","Data":"b60e64c3e6299059bd90fdd8fcb7ddf1bc077ab70499754f70fc16773377db6b"} Jul 08 19:34:02.881223 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.881204 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:34:02.882257 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.882197 2573 generic.go:356] "Generic (PLEG): container finished" podID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerID="e37d4545ca158968ce59563fb5902cfa31f0b37dd599c0182799808c41880f9f" exitCode=0 Jul 08 19:34:02.882371 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.882283 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" event={"ID":"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889","Type":"ContainerDied","Data":"e37d4545ca158968ce59563fb5902cfa31f0b37dd599c0182799808c41880f9f"} Jul 08 19:34:02.882371 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.882335 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" event={"ID":"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889","Type":"ContainerStarted","Data":"da138a2e3a82548f76344b0f12185ce92c8817deed597959e4a3e78e0e9ecd96"} Jul 08 19:34:02.909484 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.909453 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w"] Jul 08 19:34:02.912042 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.911999 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:02.922723 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.922694 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w"] Jul 08 19:34:02.953235 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.953196 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzrtr\" (UniqueName: \"kubernetes.io/projected/30374b44-9297-43c7-b286-71ec0955811f-kube-api-access-mzrtr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:02.953412 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.953247 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:02.953412 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:02.953322 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:03.054292 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.054245 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzrtr\" (UniqueName: \"kubernetes.io/projected/30374b44-9297-43c7-b286-71ec0955811f-kube-api-access-mzrtr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:03.054467 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.054309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:03.054467 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.054377 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:03.054834 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.054809 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:03.054879 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.054822 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:03.062744 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.062713 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzrtr\" (UniqueName: \"kubernetes.io/projected/30374b44-9297-43c7-b286-71ec0955811f-kube-api-access-mzrtr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:03.228403 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.228318 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:03.302516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.302482 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25"] Jul 08 19:34:03.305344 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.305323 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.315980 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.315567 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25"] Jul 08 19:34:03.356341 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.356307 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.356481 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.356361 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.356481 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.356433 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54hq8\" (UniqueName: \"kubernetes.io/projected/1b89c29e-1808-453d-8cbe-f356a0498d7d-kube-api-access-54hq8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.367566 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.367414 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w"] Jul 08 19:34:03.369618 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:03.369592 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30374b44_9297_43c7_b286_71ec0955811f.slice/crio-06bb5fd58c69c784407c60f6c66b0fbb9b6faa3578dd93d5cabc788056135fa6 WatchSource:0}: Error finding container 06bb5fd58c69c784407c60f6c66b0fbb9b6faa3578dd93d5cabc788056135fa6: Status 404 returned error can't find the container with id 06bb5fd58c69c784407c60f6c66b0fbb9b6faa3578dd93d5cabc788056135fa6 Jul 08 19:34:03.456864 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.456837 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.457043 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.456901 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.457043 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.456937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54hq8\" (UniqueName: \"kubernetes.io/projected/1b89c29e-1808-453d-8cbe-f356a0498d7d-kube-api-access-54hq8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.457313 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.457290 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.457376 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.457307 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.466067 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.466045 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54hq8\" (UniqueName: \"kubernetes.io/projected/1b89c29e-1808-453d-8cbe-f356a0498d7d-kube-api-access-54hq8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.616178 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.616145 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:03.874122 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.874054 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25"] Jul 08 19:34:03.888953 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.888925 2573 generic.go:356] "Generic (PLEG): container finished" podID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerID="6742d9b1195e9d6c0039685233f38d459e6821b712a06f8a3c31c6c5b7bbe395" exitCode=0 Jul 08 19:34:03.889118 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.889009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" event={"ID":"b47e78cb-e83e-46a8-b391-d15363b90f9f","Type":"ContainerDied","Data":"6742d9b1195e9d6c0039685233f38d459e6821b712a06f8a3c31c6c5b7bbe395"} Jul 08 19:34:03.890868 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.890849 2573 generic.go:356] "Generic (PLEG): container finished" podID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerID="e55bbce4b08c5ef633af109d5aabcaca052ada4c3ba936b022ffefe90fcf0701" exitCode=0 Jul 08 19:34:03.890949 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.890934 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" event={"ID":"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889","Type":"ContainerDied","Data":"e55bbce4b08c5ef633af109d5aabcaca052ada4c3ba936b022ffefe90fcf0701"} Jul 08 19:34:03.892562 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.892539 2573 generic.go:356] "Generic (PLEG): container finished" podID="30374b44-9297-43c7-b286-71ec0955811f" containerID="474aaff2a044b265ad7d8a56b44a99f8d6b969bdd65ed3436b76d0a316c05646" exitCode=0 Jul 08 19:34:03.892653 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.892598 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" event={"ID":"30374b44-9297-43c7-b286-71ec0955811f","Type":"ContainerDied","Data":"474aaff2a044b265ad7d8a56b44a99f8d6b969bdd65ed3436b76d0a316c05646"} Jul 08 19:34:03.892705 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:03.892688 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" event={"ID":"30374b44-9297-43c7-b286-71ec0955811f","Type":"ContainerStarted","Data":"06bb5fd58c69c784407c60f6c66b0fbb9b6faa3578dd93d5cabc788056135fa6"} Jul 08 19:34:03.895766 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:03.895742 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b89c29e_1808_453d_8cbe_f356a0498d7d.slice/crio-e7aeac05dd3ebb4f9153e2bd01cc354ab86986d374fbe46e6a8796dd331362e5 WatchSource:0}: Error finding container e7aeac05dd3ebb4f9153e2bd01cc354ab86986d374fbe46e6a8796dd331362e5: Status 404 returned error can't find the container with id e7aeac05dd3ebb4f9153e2bd01cc354ab86986d374fbe46e6a8796dd331362e5 Jul 08 19:34:04.900031 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.899998 2573 generic.go:356] "Generic (PLEG): container finished" podID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerID="abb1f12db81213e18164558034d2a3ac3ea153e6ab9fed25831f2193b5128bee" exitCode=0 Jul 08 19:34:04.900476 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.900085 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" event={"ID":"b47e78cb-e83e-46a8-b391-d15363b90f9f","Type":"ContainerDied","Data":"abb1f12db81213e18164558034d2a3ac3ea153e6ab9fed25831f2193b5128bee"} Jul 08 19:34:04.902050 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.902008 2573 generic.go:356] "Generic (PLEG): container finished" podID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerID="99144cf08d3e1ab63e5095f8fa03cd75a01342a8db41a113613e692c8c7999d9" exitCode=0 Jul 08 19:34:04.902148 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.902048 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" event={"ID":"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889","Type":"ContainerDied","Data":"99144cf08d3e1ab63e5095f8fa03cd75a01342a8db41a113613e692c8c7999d9"} Jul 08 19:34:04.903431 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.903408 2573 generic.go:356] "Generic (PLEG): container finished" podID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerID="51993555cd63381dfe7fb57dcc3020852599a20a62fcaee81920063eddf6eb7d" exitCode=0 Jul 08 19:34:04.903531 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.903487 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" event={"ID":"1b89c29e-1808-453d-8cbe-f356a0498d7d","Type":"ContainerDied","Data":"51993555cd63381dfe7fb57dcc3020852599a20a62fcaee81920063eddf6eb7d"} Jul 08 19:34:04.903595 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.903547 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" event={"ID":"1b89c29e-1808-453d-8cbe-f356a0498d7d","Type":"ContainerStarted","Data":"e7aeac05dd3ebb4f9153e2bd01cc354ab86986d374fbe46e6a8796dd331362e5"} Jul 08 19:34:04.905381 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.905354 2573 generic.go:356] "Generic (PLEG): container finished" podID="30374b44-9297-43c7-b286-71ec0955811f" containerID="f0cf9e7be1bafffd753849c96a1dbcf7822488b8215b3f67835b527638827c38" exitCode=0 Jul 08 19:34:04.905477 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:04.905412 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" event={"ID":"30374b44-9297-43c7-b286-71ec0955811f","Type":"ContainerDied","Data":"f0cf9e7be1bafffd753849c96a1dbcf7822488b8215b3f67835b527638827c38"} Jul 08 19:34:05.912238 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:05.912148 2573 generic.go:356] "Generic (PLEG): container finished" podID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerID="c9690ce1d3797656504f3ec8ba6b8956a88dbeabce2b571422dec88484f7c2b0" exitCode=0 Jul 08 19:34:05.912612 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:05.912235 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" event={"ID":"1b89c29e-1808-453d-8cbe-f356a0498d7d","Type":"ContainerDied","Data":"c9690ce1d3797656504f3ec8ba6b8956a88dbeabce2b571422dec88484f7c2b0"} Jul 08 19:34:05.914383 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:05.914366 2573 generic.go:356] "Generic (PLEG): container finished" podID="30374b44-9297-43c7-b286-71ec0955811f" containerID="58c0e8861c1c927ea478753fba841d3bb6ac25261f0958de4d8fbbae8043da7d" exitCode=0 Jul 08 19:34:05.914474 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:05.914450 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" event={"ID":"30374b44-9297-43c7-b286-71ec0955811f","Type":"ContainerDied","Data":"58c0e8861c1c927ea478753fba841d3bb6ac25261f0958de4d8fbbae8043da7d"} Jul 08 19:34:06.111967 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.111942 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:06.118918 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.118895 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:06.183764 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.183679 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9srqk\" (UniqueName: \"kubernetes.io/projected/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-kube-api-access-9srqk\") pod \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " Jul 08 19:34:06.183764 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.183766 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsfpl\" (UniqueName: \"kubernetes.io/projected/b47e78cb-e83e-46a8-b391-d15363b90f9f-kube-api-access-dsfpl\") pod \"b47e78cb-e83e-46a8-b391-d15363b90f9f\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " Jul 08 19:34:06.183992 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.183800 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-bundle\") pod \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " Jul 08 19:34:06.183992 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.183817 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-util\") pod \"b47e78cb-e83e-46a8-b391-d15363b90f9f\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " Jul 08 19:34:06.183992 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.183832 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-util\") pod \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\" (UID: \"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889\") " Jul 08 19:34:06.183992 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.183849 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-bundle\") pod \"b47e78cb-e83e-46a8-b391-d15363b90f9f\" (UID: \"b47e78cb-e83e-46a8-b391-d15363b90f9f\") " Jul 08 19:34:06.184486 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.184419 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-bundle" (OuterVolumeSpecName: "bundle") pod "b47e78cb-e83e-46a8-b391-d15363b90f9f" (UID: "b47e78cb-e83e-46a8-b391-d15363b90f9f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:06.184486 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.184467 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-bundle" (OuterVolumeSpecName: "bundle") pod "cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" (UID: "cbfe2a9d-2ab8-4df8-b52d-a36fc2616889"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:06.186345 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.186318 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-kube-api-access-9srqk" (OuterVolumeSpecName: "kube-api-access-9srqk") pod "cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" (UID: "cbfe2a9d-2ab8-4df8-b52d-a36fc2616889"). InnerVolumeSpecName "kube-api-access-9srqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:06.186454 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.186342 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b47e78cb-e83e-46a8-b391-d15363b90f9f-kube-api-access-dsfpl" (OuterVolumeSpecName: "kube-api-access-dsfpl") pod "b47e78cb-e83e-46a8-b391-d15363b90f9f" (UID: "b47e78cb-e83e-46a8-b391-d15363b90f9f"). InnerVolumeSpecName "kube-api-access-dsfpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:06.189945 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.189921 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-util" (OuterVolumeSpecName: "util") pod "b47e78cb-e83e-46a8-b391-d15363b90f9f" (UID: "b47e78cb-e83e-46a8-b391-d15363b90f9f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:06.190368 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.190347 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-util" (OuterVolumeSpecName: "util") pod "cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" (UID: "cbfe2a9d-2ab8-4df8-b52d-a36fc2616889"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:06.284967 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.284907 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsfpl\" (UniqueName: \"kubernetes.io/projected/b47e78cb-e83e-46a8-b391-d15363b90f9f-kube-api-access-dsfpl\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:06.284967 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.284959 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:06.284967 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.284970 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-util\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:06.284967 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.284978 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-util\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:06.284967 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.284986 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b47e78cb-e83e-46a8-b391-d15363b90f9f-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:06.285311 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.284995 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9srqk\" (UniqueName: \"kubernetes.io/projected/cbfe2a9d-2ab8-4df8-b52d-a36fc2616889-kube-api-access-9srqk\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:06.921242 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.921144 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" Jul 08 19:34:06.921242 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.921135 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p" event={"ID":"cbfe2a9d-2ab8-4df8-b52d-a36fc2616889","Type":"ContainerDied","Data":"da138a2e3a82548f76344b0f12185ce92c8817deed597959e4a3e78e0e9ecd96"} Jul 08 19:34:06.921717 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.921276 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da138a2e3a82548f76344b0f12185ce92c8817deed597959e4a3e78e0e9ecd96" Jul 08 19:34:06.923363 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.923331 2573 generic.go:356] "Generic (PLEG): container finished" podID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerID="12d43f65e87d35a9329931d43395249b64fc2827eef793a3b8a208d5da2fa071" exitCode=0 Jul 08 19:34:06.923500 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.923464 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" event={"ID":"1b89c29e-1808-453d-8cbe-f356a0498d7d","Type":"ContainerDied","Data":"12d43f65e87d35a9329931d43395249b64fc2827eef793a3b8a208d5da2fa071"} Jul 08 19:34:06.925409 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.925380 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" event={"ID":"b47e78cb-e83e-46a8-b391-d15363b90f9f","Type":"ContainerDied","Data":"b60e64c3e6299059bd90fdd8fcb7ddf1bc077ab70499754f70fc16773377db6b"} Jul 08 19:34:06.925409 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.925399 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l" Jul 08 19:34:06.925590 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:06.925425 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b60e64c3e6299059bd90fdd8fcb7ddf1bc077ab70499754f70fc16773377db6b" Jul 08 19:34:07.054333 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.054309 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:07.093910 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.093876 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzrtr\" (UniqueName: \"kubernetes.io/projected/30374b44-9297-43c7-b286-71ec0955811f-kube-api-access-mzrtr\") pod \"30374b44-9297-43c7-b286-71ec0955811f\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " Jul 08 19:34:07.094084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.093985 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-util\") pod \"30374b44-9297-43c7-b286-71ec0955811f\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " Jul 08 19:34:07.094084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.094015 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-bundle\") pod \"30374b44-9297-43c7-b286-71ec0955811f\" (UID: \"30374b44-9297-43c7-b286-71ec0955811f\") " Jul 08 19:34:07.094764 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.094730 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-bundle" (OuterVolumeSpecName: "bundle") pod "30374b44-9297-43c7-b286-71ec0955811f" (UID: "30374b44-9297-43c7-b286-71ec0955811f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:07.095913 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.095886 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30374b44-9297-43c7-b286-71ec0955811f-kube-api-access-mzrtr" (OuterVolumeSpecName: "kube-api-access-mzrtr") pod "30374b44-9297-43c7-b286-71ec0955811f" (UID: "30374b44-9297-43c7-b286-71ec0955811f"). InnerVolumeSpecName "kube-api-access-mzrtr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:07.099609 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.099587 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-util" (OuterVolumeSpecName: "util") pod "30374b44-9297-43c7-b286-71ec0955811f" (UID: "30374b44-9297-43c7-b286-71ec0955811f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:07.194953 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.194857 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-util\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:07.194953 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.194892 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30374b44-9297-43c7-b286-71ec0955811f-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:07.194953 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.194905 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzrtr\" (UniqueName: \"kubernetes.io/projected/30374b44-9297-43c7-b286-71ec0955811f-kube-api-access-mzrtr\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:07.932528 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.932497 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" Jul 08 19:34:07.933008 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.932532 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w" event={"ID":"30374b44-9297-43c7-b286-71ec0955811f","Type":"ContainerDied","Data":"06bb5fd58c69c784407c60f6c66b0fbb9b6faa3578dd93d5cabc788056135fa6"} Jul 08 19:34:07.933008 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:07.932579 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06bb5fd58c69c784407c60f6c66b0fbb9b6faa3578dd93d5cabc788056135fa6" Jul 08 19:34:08.068547 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.068521 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:08.101179 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.101146 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54hq8\" (UniqueName: \"kubernetes.io/projected/1b89c29e-1808-453d-8cbe-f356a0498d7d-kube-api-access-54hq8\") pod \"1b89c29e-1808-453d-8cbe-f356a0498d7d\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " Jul 08 19:34:08.101345 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.101204 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-util\") pod \"1b89c29e-1808-453d-8cbe-f356a0498d7d\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " Jul 08 19:34:08.101345 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.101237 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-bundle\") pod \"1b89c29e-1808-453d-8cbe-f356a0498d7d\" (UID: \"1b89c29e-1808-453d-8cbe-f356a0498d7d\") " Jul 08 19:34:08.101763 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.101737 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-bundle" (OuterVolumeSpecName: "bundle") pod "1b89c29e-1808-453d-8cbe-f356a0498d7d" (UID: "1b89c29e-1808-453d-8cbe-f356a0498d7d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:08.103487 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.103453 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b89c29e-1808-453d-8cbe-f356a0498d7d-kube-api-access-54hq8" (OuterVolumeSpecName: "kube-api-access-54hq8") pod "1b89c29e-1808-453d-8cbe-f356a0498d7d" (UID: "1b89c29e-1808-453d-8cbe-f356a0498d7d"). InnerVolumeSpecName "kube-api-access-54hq8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:08.110334 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.110303 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-util" (OuterVolumeSpecName: "util") pod "1b89c29e-1808-453d-8cbe-f356a0498d7d" (UID: "1b89c29e-1808-453d-8cbe-f356a0498d7d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:08.202579 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.202490 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54hq8\" (UniqueName: \"kubernetes.io/projected/1b89c29e-1808-453d-8cbe-f356a0498d7d-kube-api-access-54hq8\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:08.202579 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.202521 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-util\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:08.202579 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.202536 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b89c29e-1808-453d-8cbe-f356a0498d7d-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:08.939361 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.939262 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" event={"ID":"1b89c29e-1808-453d-8cbe-f356a0498d7d","Type":"ContainerDied","Data":"e7aeac05dd3ebb4f9153e2bd01cc354ab86986d374fbe46e6a8796dd331362e5"} Jul 08 19:34:08.939361 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.939301 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25" Jul 08 19:34:08.939361 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:08.939353 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7aeac05dd3ebb4f9153e2bd01cc354ab86986d374fbe46e6a8796dd331362e5" Jul 08 19:34:24.996731 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.996641 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ttps6"] Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997073 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerName="pull" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997090 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerName="pull" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997101 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerName="pull" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997107 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerName="pull" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997114 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerName="util" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997119 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerName="util" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997127 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30374b44-9297-43c7-b286-71ec0955811f" containerName="pull" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997132 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30374b44-9297-43c7-b286-71ec0955811f" containerName="pull" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997140 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerName="extract" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997148 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerName="extract" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997154 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerName="util" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997159 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerName="util" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997170 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30374b44-9297-43c7-b286-71ec0955811f" containerName="extract" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997175 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30374b44-9297-43c7-b286-71ec0955811f" containerName="extract" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997184 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerName="pull" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997190 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerName="pull" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997195 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerName="extract" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997201 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerName="extract" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997206 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerName="extract" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997212 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerName="extract" Jul 08 19:34:24.997210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997220 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerName="util" Jul 08 19:34:24.997849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997225 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerName="util" Jul 08 19:34:24.997849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997231 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30374b44-9297-43c7-b286-71ec0955811f" containerName="util" Jul 08 19:34:24.997849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997236 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30374b44-9297-43c7-b286-71ec0955811f" containerName="util" Jul 08 19:34:24.997849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997290 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbfe2a9d-2ab8-4df8-b52d-a36fc2616889" containerName="extract" Jul 08 19:34:24.997849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997298 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b47e78cb-e83e-46a8-b391-d15363b90f9f" containerName="extract" Jul 08 19:34:24.997849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997306 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b89c29e-1808-453d-8cbe-f356a0498d7d" containerName="extract" Jul 08 19:34:24.997849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:24.997312 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="30374b44-9297-43c7-b286-71ec0955811f" containerName="extract" Jul 08 19:34:25.003536 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:25.003513 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" Jul 08 19:34:25.006066 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:25.006040 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qknvv\"" Jul 08 19:34:25.009544 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:25.009510 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ttps6"] Jul 08 19:34:25.153602 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:25.153562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dms5\" (UniqueName: \"kubernetes.io/projected/0f2532d9-bf2c-48e7-877e-c3384f674cc4-kube-api-access-6dms5\") pod \"authorino-operator-75cc8d5cb7-ttps6\" (UID: \"0f2532d9-bf2c-48e7-877e-c3384f674cc4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" Jul 08 19:34:25.254129 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:25.254002 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dms5\" (UniqueName: \"kubernetes.io/projected/0f2532d9-bf2c-48e7-877e-c3384f674cc4-kube-api-access-6dms5\") pod \"authorino-operator-75cc8d5cb7-ttps6\" (UID: \"0f2532d9-bf2c-48e7-877e-c3384f674cc4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" Jul 08 19:34:25.262862 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:25.262835 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dms5\" (UniqueName: \"kubernetes.io/projected/0f2532d9-bf2c-48e7-877e-c3384f674cc4-kube-api-access-6dms5\") pod \"authorino-operator-75cc8d5cb7-ttps6\" (UID: \"0f2532d9-bf2c-48e7-877e-c3384f674cc4\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" Jul 08 19:34:25.316070 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:25.315995 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" Jul 08 19:34:25.451287 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:25.451262 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-ttps6"] Jul 08 19:34:25.453347 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:25.453318 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2532d9_bf2c_48e7_877e_c3384f674cc4.slice/crio-23d189c221cc476c4c8695954ccedfb04cf0da3cd6e79e9e9ef0d802981427db WatchSource:0}: Error finding container 23d189c221cc476c4c8695954ccedfb04cf0da3cd6e79e9e9ef0d802981427db: Status 404 returned error can't find the container with id 23d189c221cc476c4c8695954ccedfb04cf0da3cd6e79e9e9ef0d802981427db Jul 08 19:34:26.019709 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.019674 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" event={"ID":"0f2532d9-bf2c-48e7-877e-c3384f674cc4","Type":"ContainerStarted","Data":"23d189c221cc476c4c8695954ccedfb04cf0da3cd6e79e9e9ef0d802981427db"} Jul 08 19:34:26.280628 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.280537 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-789756545c-f752n"] Jul 08 19:34:26.284327 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.284304 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.297495 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.297466 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-789756545c-f752n"] Jul 08 19:34:26.364522 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.364481 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-oauth-serving-cert\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.364696 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.364527 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-oauth-config\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.364696 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.364556 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-serving-cert\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.364696 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.364616 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-trusted-ca-bundle\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.364696 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.364662 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-config\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.364696 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.364688 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-service-ca\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.364972 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.364717 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p5fc\" (UniqueName: \"kubernetes.io/projected/68106ee9-eb9f-46ca-bc04-44371467d9cc-kube-api-access-4p5fc\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.465827 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-trusted-ca-bundle\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.465984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-config\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.466081 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-service-ca\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.466119 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p5fc\" (UniqueName: \"kubernetes.io/projected/68106ee9-eb9f-46ca-bc04-44371467d9cc-kube-api-access-4p5fc\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.466162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-oauth-serving-cert\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.466186 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-oauth-config\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.466213 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-serving-cert\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.466609 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-config\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.467702 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-oauth-serving-cert\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.468521 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-trusted-ca-bundle\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.468584 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68106ee9-eb9f-46ca-bc04-44371467d9cc-service-ca\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.472216 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.470692 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-oauth-config\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.474812 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.474784 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68106ee9-eb9f-46ca-bc04-44371467d9cc-console-serving-cert\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.478095 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.478070 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p5fc\" (UniqueName: \"kubernetes.io/projected/68106ee9-eb9f-46ca-bc04-44371467d9cc-kube-api-access-4p5fc\") pod \"console-789756545c-f752n\" (UID: \"68106ee9-eb9f-46ca-bc04-44371467d9cc\") " pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.596306 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.596213 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:26.765414 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:26.765378 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-789756545c-f752n"] Jul 08 19:34:26.770866 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:26.770834 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68106ee9_eb9f_46ca_bc04_44371467d9cc.slice/crio-02ff6e11f44043f7c4cba979d8a8bbc25697db0bf1d559f585b4315885755872 WatchSource:0}: Error finding container 02ff6e11f44043f7c4cba979d8a8bbc25697db0bf1d559f585b4315885755872: Status 404 returned error can't find the container with id 02ff6e11f44043f7c4cba979d8a8bbc25697db0bf1d559f585b4315885755872 Jul 08 19:34:27.027945 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:27.027905 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-789756545c-f752n" event={"ID":"68106ee9-eb9f-46ca-bc04-44371467d9cc","Type":"ContainerStarted","Data":"02ff6e11f44043f7c4cba979d8a8bbc25697db0bf1d559f585b4315885755872"} Jul 08 19:34:28.035083 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.034965 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" event={"ID":"0f2532d9-bf2c-48e7-877e-c3384f674cc4","Type":"ContainerStarted","Data":"503d7f1975b805203ba1d4187fb977156d82d3e5d3056b3fc9b1719ab325e5f6"} Jul 08 19:34:28.035540 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.035082 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" Jul 08 19:34:28.036896 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.036869 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-789756545c-f752n" event={"ID":"68106ee9-eb9f-46ca-bc04-44371467d9cc","Type":"ContainerStarted","Data":"7aa6d23615bc181a91176ccb389fa23a7c16af3643d3e349efa69a5dafa4100c"} Jul 08 19:34:28.055589 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.055520 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" podStartSLOduration=1.711748417 podStartE2EDuration="4.055503385s" totalImagesPullingTime="2.343754968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:28.053160563 +0000 UTC m=+629.710599197" watchObservedRunningTime="2026-07-08 19:34:28.055503385 +0000 UTC m=+629.712941991" Jul 08 19:34:28.072279 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.072213 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-789756545c-f752n" podStartSLOduration=2.07219397 podStartE2EDuration="2.07219397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:28.070880019 +0000 UTC m=+629.728318622" watchObservedRunningTime="2026-07-08 19:34:28.07219397 +0000 UTC m=+629.729632578" Jul 08 19:34:28.447041 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.446991 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9"] Jul 08 19:34:28.450328 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.450307 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" Jul 08 19:34:28.452878 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.452818 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 08 19:34:28.453089 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.452913 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4l728\"" Jul 08 19:34:28.462532 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.462508 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9"] Jul 08 19:34:28.583203 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.583171 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brw5j\" (UniqueName: \"kubernetes.io/projected/6b9542bb-3547-4c88-b926-5445a3279e34-kube-api-access-brw5j\") pod \"dns-operator-controller-manager-864dd78c4c-7kjk9\" (UID: \"6b9542bb-3547-4c88-b926-5445a3279e34\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" Jul 08 19:34:28.683877 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.683786 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brw5j\" (UniqueName: \"kubernetes.io/projected/6b9542bb-3547-4c88-b926-5445a3279e34-kube-api-access-brw5j\") pod \"dns-operator-controller-manager-864dd78c4c-7kjk9\" (UID: \"6b9542bb-3547-4c88-b926-5445a3279e34\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" Jul 08 19:34:28.694371 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.694343 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brw5j\" (UniqueName: \"kubernetes.io/projected/6b9542bb-3547-4c88-b926-5445a3279e34-kube-api-access-brw5j\") pod \"dns-operator-controller-manager-864dd78c4c-7kjk9\" (UID: \"6b9542bb-3547-4c88-b926-5445a3279e34\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" Jul 08 19:34:28.760327 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.760288 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" Jul 08 19:34:28.898336 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:28.898310 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9"] Jul 08 19:34:28.900697 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:28.900670 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b9542bb_3547_4c88_b926_5445a3279e34.slice/crio-3c131d465138dd0e0d24ae2428be7b1eb48967a49223a6d8d799ca47a0d1fecd WatchSource:0}: Error finding container 3c131d465138dd0e0d24ae2428be7b1eb48967a49223a6d8d799ca47a0d1fecd: Status 404 returned error can't find the container with id 3c131d465138dd0e0d24ae2428be7b1eb48967a49223a6d8d799ca47a0d1fecd Jul 08 19:34:29.043170 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:29.043136 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" event={"ID":"6b9542bb-3547-4c88-b926-5445a3279e34","Type":"ContainerStarted","Data":"3c131d465138dd0e0d24ae2428be7b1eb48967a49223a6d8d799ca47a0d1fecd"} Jul 08 19:34:32.062325 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:32.062287 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" event={"ID":"6b9542bb-3547-4c88-b926-5445a3279e34","Type":"ContainerStarted","Data":"501d59e02e44ab5f94f4e83e0334d5e6a253f2dc8db6c3a64111c3b8f1c1bdc2"} Jul 08 19:34:32.062822 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:32.062414 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" Jul 08 19:34:32.079860 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:32.079811 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" podStartSLOduration=1.843103382 podStartE2EDuration="4.079794852s" totalImagesPullingTime="2.23669147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:32.07862585 +0000 UTC m=+633.736064479" watchObservedRunningTime="2026-07-08 19:34:32.079794852 +0000 UTC m=+633.737233458" Jul 08 19:34:33.360984 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.360943 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw"] Jul 08 19:34:33.364593 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.364571 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:33.367151 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.367126 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jx8qd\"" Jul 08 19:34:33.374189 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.374157 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw"] Jul 08 19:34:33.526468 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.526427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28sk9\" (UniqueName: \"kubernetes.io/projected/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-kube-api-access-28sk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" (UID: \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:33.526645 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.526530 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" (UID: \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:33.627409 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.627301 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28sk9\" (UniqueName: \"kubernetes.io/projected/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-kube-api-access-28sk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" (UID: \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:33.627409 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.627404 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" (UID: \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:33.627811 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.627789 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" (UID: \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:33.635597 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.635563 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28sk9\" (UniqueName: \"kubernetes.io/projected/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-kube-api-access-28sk9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" (UID: \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:33.676588 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.676545 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:33.820551 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:33.820522 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw"] Jul 08 19:34:33.822820 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:33.822795 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88ac38e_1ee0_4f7c_b22a_1ad5f3daf186.slice/crio-2114a28d4063f44f4d43d6ac82721a80f8ff45bc13e3026afbc8f2f8cde39612 WatchSource:0}: Error finding container 2114a28d4063f44f4d43d6ac82721a80f8ff45bc13e3026afbc8f2f8cde39612: Status 404 returned error can't find the container with id 2114a28d4063f44f4d43d6ac82721a80f8ff45bc13e3026afbc8f2f8cde39612 Jul 08 19:34:34.073230 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:34.073188 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" event={"ID":"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186","Type":"ContainerStarted","Data":"2114a28d4063f44f4d43d6ac82721a80f8ff45bc13e3026afbc8f2f8cde39612"} Jul 08 19:34:35.619114 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:35.619076 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2"] Jul 08 19:34:35.623232 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:35.623201 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:35.625817 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:35.625730 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5pvnt\"" Jul 08 19:34:35.632918 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:35.632889 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2"] Jul 08 19:34:35.747118 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:35.747076 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnqct\" (UniqueName: \"kubernetes.io/projected/be8db7a0-01e0-48a3-83d2-879f86559604-kube-api-access-hnqct\") pod \"limitador-operator-controller-manager-859d7b78bd-gtkb2\" (UID: \"be8db7a0-01e0-48a3-83d2-879f86559604\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:35.848467 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:35.848430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnqct\" (UniqueName: \"kubernetes.io/projected/be8db7a0-01e0-48a3-83d2-879f86559604-kube-api-access-hnqct\") pod \"limitador-operator-controller-manager-859d7b78bd-gtkb2\" (UID: \"be8db7a0-01e0-48a3-83d2-879f86559604\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:35.860274 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:35.858492 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnqct\" (UniqueName: \"kubernetes.io/projected/be8db7a0-01e0-48a3-83d2-879f86559604-kube-api-access-hnqct\") pod \"limitador-operator-controller-manager-859d7b78bd-gtkb2\" (UID: \"be8db7a0-01e0-48a3-83d2-879f86559604\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:35.937469 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:35.937379 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:36.098163 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:36.098132 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2"] Jul 08 19:34:36.101411 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:36.101377 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8db7a0_01e0_48a3_83d2_879f86559604.slice/crio-401d8fff44be366a62e9fbc31954d3cbc31c99eb2cbd0ae6936c131a6c81250d WatchSource:0}: Error finding container 401d8fff44be366a62e9fbc31954d3cbc31c99eb2cbd0ae6936c131a6c81250d: Status 404 returned error can't find the container with id 401d8fff44be366a62e9fbc31954d3cbc31c99eb2cbd0ae6936c131a6c81250d Jul 08 19:34:36.596586 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:36.596546 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:36.596780 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:36.596625 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:36.602331 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:36.602301 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:37.091808 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:37.091774 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" event={"ID":"be8db7a0-01e0-48a3-83d2-879f86559604","Type":"ContainerStarted","Data":"401d8fff44be366a62e9fbc31954d3cbc31c99eb2cbd0ae6936c131a6c81250d"} Jul 08 19:34:37.095988 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:37.095963 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-789756545c-f752n" Jul 08 19:34:37.149997 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:37.149960 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7897b8dd64-5d78v"] Jul 08 19:34:39.046329 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:39.046300 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-ttps6" Jul 08 19:34:39.108526 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:39.108480 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" event={"ID":"be8db7a0-01e0-48a3-83d2-879f86559604","Type":"ContainerStarted","Data":"06ced26d2329fb00f59440c5b76ac754a2895d2bc1b3f075c942949ecac73bf0"} Jul 08 19:34:39.108711 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:39.108570 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:39.127106 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:39.127007 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" podStartSLOduration=2.205419139 podStartE2EDuration="4.12698942s" totalImagesPullingTime="1.921570281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:39.124940399 +0000 UTC m=+640.782379025" watchObservedRunningTime="2026-07-08 19:34:39.12698942 +0000 UTC m=+640.784428026" Jul 08 19:34:41.120799 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:41.120760 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" event={"ID":"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186","Type":"ContainerStarted","Data":"05c8690a351ff53aad1f72253584752eb2d2cc54d81d8692c44af943e5a17565"} Jul 08 19:34:41.121305 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:41.120872 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:41.141489 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:41.141431 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" podStartSLOduration=1.530399451 podStartE2EDuration="8.141414217s" totalImagesPullingTime="6.611014766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:41.139118752 +0000 UTC m=+642.796557371" watchObservedRunningTime="2026-07-08 19:34:41.141414217 +0000 UTC m=+642.798852824" Jul 08 19:34:43.069061 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:43.069015 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7kjk9" Jul 08 19:34:50.116943 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:50.116912 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:52.128632 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:52.128599 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:53.929235 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.929195 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw"] Jul 08 19:34:53.929701 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.929374 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" containerName="manager" containerID="cri-o://05c8690a351ff53aad1f72253584752eb2d2cc54d81d8692c44af943e5a17565" gracePeriod=2 Jul 08 19:34:53.939286 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:34:53.939250 2573 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-45pcw\": pods \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:53.940266 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.940234 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw"] Jul 08 19:34:53.955123 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.955096 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb"] Jul 08 19:34:53.955477 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.955463 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" containerName="manager" Jul 08 19:34:53.955521 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.955479 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" containerName="manager" Jul 08 19:34:53.959045 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.955961 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" containerName="manager" Jul 08 19:34:53.962566 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.962536 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:53.964585 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.964562 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2"] Jul 08 19:34:53.964790 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.964765 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" podUID="be8db7a0-01e0-48a3-83d2-879f86559604" containerName="manager" containerID="cri-o://06ced26d2329fb00f59440c5b76ac754a2895d2bc1b3f075c942949ecac73bf0" gracePeriod=2 Jul 08 19:34:53.968178 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:34:53.968153 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" is forbidden: User \"system:node:ip-10-0-143-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-40.ec2.internal' and this object" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:53.972980 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.972956 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2"] Jul 08 19:34:53.980666 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.980637 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb"] Jul 08 19:34:53.989861 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.989835 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd"] Jul 08 19:34:53.990193 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.990180 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be8db7a0-01e0-48a3-83d2-879f86559604" containerName="manager" Jul 08 19:34:53.990247 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.990195 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="be8db7a0-01e0-48a3-83d2-879f86559604" containerName="manager" Jul 08 19:34:53.990281 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.990255 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="be8db7a0-01e0-48a3-83d2-879f86559604" containerName="manager" Jul 08 19:34:53.993458 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:53.993435 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" Jul 08 19:34:54.009111 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.008774 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fb5d39b-5528-433f-ba49-5821c50e7a98-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gfxjb\" (UID: \"8fb5d39b-5528-433f-ba49-5821c50e7a98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:54.009111 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.008888 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvl4m\" (UniqueName: \"kubernetes.io/projected/8fb5d39b-5528-433f-ba49-5821c50e7a98-kube-api-access-qvl4m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gfxjb\" (UID: \"8fb5d39b-5528-433f-ba49-5821c50e7a98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:54.009536 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:34:54.009501 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" is forbidden: User \"system:node:ip-10-0-143-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-40.ec2.internal' and this object" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:54.011727 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:34:54.011692 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gtkb2\" is forbidden: User \"system:node:ip-10-0-143-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-40.ec2.internal' and this object" podUID="be8db7a0-01e0-48a3-83d2-879f86559604" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:54.013210 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.013187 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd"] Jul 08 19:34:54.109838 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.109794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvl4m\" (UniqueName: \"kubernetes.io/projected/8fb5d39b-5528-433f-ba49-5821c50e7a98-kube-api-access-qvl4m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gfxjb\" (UID: \"8fb5d39b-5528-433f-ba49-5821c50e7a98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:54.110033 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.109866 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmsv\" (UniqueName: \"kubernetes.io/projected/2009d245-c529-4f6b-96a4-f02cf48b7904-kube-api-access-cbmsv\") pod \"limitador-operator-controller-manager-859d7b78bd-h92wd\" (UID: \"2009d245-c529-4f6b-96a4-f02cf48b7904\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" Jul 08 19:34:54.110118 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.110046 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fb5d39b-5528-433f-ba49-5821c50e7a98-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gfxjb\" (UID: \"8fb5d39b-5528-433f-ba49-5821c50e7a98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:54.110403 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.110385 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fb5d39b-5528-433f-ba49-5821c50e7a98-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gfxjb\" (UID: \"8fb5d39b-5528-433f-ba49-5821c50e7a98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:54.119964 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.119929 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvl4m\" (UniqueName: \"kubernetes.io/projected/8fb5d39b-5528-433f-ba49-5821c50e7a98-kube-api-access-qvl4m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gfxjb\" (UID: \"8fb5d39b-5528-433f-ba49-5821c50e7a98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:54.191226 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.191197 2573 generic.go:356] "Generic (PLEG): container finished" podID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" containerID="05c8690a351ff53aad1f72253584752eb2d2cc54d81d8692c44af943e5a17565" exitCode=0 Jul 08 19:34:54.191401 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.191274 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2114a28d4063f44f4d43d6ac82721a80f8ff45bc13e3026afbc8f2f8cde39612" Jul 08 19:34:54.192978 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.192952 2573 generic.go:356] "Generic (PLEG): container finished" podID="be8db7a0-01e0-48a3-83d2-879f86559604" containerID="06ced26d2329fb00f59440c5b76ac754a2895d2bc1b3f075c942949ecac73bf0" exitCode=0 Jul 08 19:34:54.200732 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.200712 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:54.202906 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:34:54.202879 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" is forbidden: User \"system:node:ip-10-0-143-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-40.ec2.internal' and this object" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:54.204080 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.204061 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:54.204674 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:34:54.204652 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gtkb2\" is forbidden: User \"system:node:ip-10-0-143-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-40.ec2.internal' and this object" podUID="be8db7a0-01e0-48a3-83d2-879f86559604" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:54.206462 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:34:54.206440 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-45pcw\" is forbidden: User \"system:node:ip-10-0-143-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-40.ec2.internal' and this object" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:54.208119 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:34:54.208097 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gtkb2\" is forbidden: User \"system:node:ip-10-0-143-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-40.ec2.internal' and this object" podUID="be8db7a0-01e0-48a3-83d2-879f86559604" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:54.210465 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.210445 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmsv\" (UniqueName: \"kubernetes.io/projected/2009d245-c529-4f6b-96a4-f02cf48b7904-kube-api-access-cbmsv\") pod \"limitador-operator-controller-manager-859d7b78bd-h92wd\" (UID: \"2009d245-c529-4f6b-96a4-f02cf48b7904\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" Jul 08 19:34:54.221661 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.221641 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmsv\" (UniqueName: \"kubernetes.io/projected/2009d245-c529-4f6b-96a4-f02cf48b7904-kube-api-access-cbmsv\") pod \"limitador-operator-controller-manager-859d7b78bd-h92wd\" (UID: \"2009d245-c529-4f6b-96a4-f02cf48b7904\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" Jul 08 19:34:54.310870 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.310837 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28sk9\" (UniqueName: \"kubernetes.io/projected/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-kube-api-access-28sk9\") pod \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\" (UID: \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\") " Jul 08 19:34:54.310870 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.310876 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-extensions-socket-volume\") pod \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\" (UID: \"f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186\") " Jul 08 19:34:54.311123 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.310925 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnqct\" (UniqueName: \"kubernetes.io/projected/be8db7a0-01e0-48a3-83d2-879f86559604-kube-api-access-hnqct\") pod \"be8db7a0-01e0-48a3-83d2-879f86559604\" (UID: \"be8db7a0-01e0-48a3-83d2-879f86559604\") " Jul 08 19:34:54.311483 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.311456 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" (UID: "f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:34:54.313072 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.313042 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-kube-api-access-28sk9" (OuterVolumeSpecName: "kube-api-access-28sk9") pod "f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" (UID: "f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186"). InnerVolumeSpecName "kube-api-access-28sk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:54.313157 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.313064 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8db7a0-01e0-48a3-83d2-879f86559604-kube-api-access-hnqct" (OuterVolumeSpecName: "kube-api-access-hnqct") pod "be8db7a0-01e0-48a3-83d2-879f86559604" (UID: "be8db7a0-01e0-48a3-83d2-879f86559604"). InnerVolumeSpecName "kube-api-access-hnqct". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:34:54.374445 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.374405 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:54.381616 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.381591 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" Jul 08 19:34:54.411983 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.411941 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28sk9\" (UniqueName: \"kubernetes.io/projected/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-kube-api-access-28sk9\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:54.412183 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.411986 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186-extensions-socket-volume\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:54.412183 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.412006 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnqct\" (UniqueName: \"kubernetes.io/projected/be8db7a0-01e0-48a3-83d2-879f86559604-kube-api-access-hnqct\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:34:54.524887 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.524861 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb"] Jul 08 19:34:54.526397 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:54.526365 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb5d39b_5528_433f_ba49_5821c50e7a98.slice/crio-746f15738e5ea9c49fb29e2c0d228fe47bb850832c70ba68a921e1109f723a99 WatchSource:0}: Error finding container 746f15738e5ea9c49fb29e2c0d228fe47bb850832c70ba68a921e1109f723a99: Status 404 returned error can't find the container with id 746f15738e5ea9c49fb29e2c0d228fe47bb850832c70ba68a921e1109f723a99 Jul 08 19:34:54.552392 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.552370 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd"] Jul 08 19:34:54.566826 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.566795 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be8db7a0-01e0-48a3-83d2-879f86559604" path="/var/lib/kubelet/pods/be8db7a0-01e0-48a3-83d2-879f86559604/volumes" Jul 08 19:34:54.567218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:54.567203 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186" path="/var/lib/kubelet/pods/f88ac38e-1ee0-4f7c-b22a-1ad5f3daf186/volumes" Jul 08 19:34:54.568422 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:34:54.568391 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2009d245_c529_4f6b_96a4_f02cf48b7904.slice/crio-525fa3535b6c723851935bf82a66ab2952a47123152aabe56f404cdb47114a7b WatchSource:0}: Error finding container 525fa3535b6c723851935bf82a66ab2952a47123152aabe56f404cdb47114a7b: Status 404 returned error can't find the container with id 525fa3535b6c723851935bf82a66ab2952a47123152aabe56f404cdb47114a7b Jul 08 19:34:55.199621 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.199583 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" event={"ID":"2009d245-c529-4f6b-96a4-f02cf48b7904","Type":"ContainerStarted","Data":"13b6ae09ef520c5852eac31ecc9f73e1444e2e6d3c574c9f788363a887d16902"} Jul 08 19:34:55.200146 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.199644 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" event={"ID":"2009d245-c529-4f6b-96a4-f02cf48b7904","Type":"ContainerStarted","Data":"525fa3535b6c723851935bf82a66ab2952a47123152aabe56f404cdb47114a7b"} Jul 08 19:34:55.200146 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.199751 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" Jul 08 19:34:55.201467 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.201398 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" event={"ID":"8fb5d39b-5528-433f-ba49-5821c50e7a98","Type":"ContainerStarted","Data":"988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9"} Jul 08 19:34:55.201467 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.201455 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" event={"ID":"8fb5d39b-5528-433f-ba49-5821c50e7a98","Type":"ContainerStarted","Data":"746f15738e5ea9c49fb29e2c0d228fe47bb850832c70ba68a921e1109f723a99"} Jul 08 19:34:55.201648 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.201521 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:34:55.202682 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.202658 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gtkb2" Jul 08 19:34:55.202682 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.202668 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-45pcw" Jul 08 19:34:55.202810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.202662 2573 scope.go:117] "RemoveContainer" containerID="06ced26d2329fb00f59440c5b76ac754a2895d2bc1b3f075c942949ecac73bf0" Jul 08 19:34:55.223337 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.223283 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" podStartSLOduration=2.223265923 podStartE2EDuration="2.223265923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:55.221016634 +0000 UTC m=+656.878455241" watchObservedRunningTime="2026-07-08 19:34:55.223265923 +0000 UTC m=+656.880704529" Jul 08 19:34:55.240280 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:34:55.240228 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" podStartSLOduration=2.240212677 podStartE2EDuration="2.240212677s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:34:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:34:55.23833008 +0000 UTC m=+656.895768685" watchObservedRunningTime="2026-07-08 19:34:55.240212677 +0000 UTC m=+656.897651282" Jul 08 19:35:02.178196 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.178149 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7897b8dd64-5d78v" podUID="e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" containerName="console" containerID="cri-o://e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78" gracePeriod=15 Jul 08 19:35:02.429521 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.429461 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:35:02.479175 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479145 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-config\") pod \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " Jul 08 19:35:02.479337 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479197 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-service-ca\") pod \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " Jul 08 19:35:02.479337 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479223 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvhw8\" (UniqueName: \"kubernetes.io/projected/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-kube-api-access-cvhw8\") pod \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " Jul 08 19:35:02.479435 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479345 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-oauth-config\") pod \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " Jul 08 19:35:02.479435 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479401 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-oauth-serving-cert\") pod \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " Jul 08 19:35:02.479435 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479430 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-serving-cert\") pod \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " Jul 08 19:35:02.479640 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479493 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-trusted-ca-bundle\") pod \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\" (UID: \"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5\") " Jul 08 19:35:02.479775 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479743 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-config" (OuterVolumeSpecName: "console-config") pod "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" (UID: "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:35:02.479775 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479755 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-service-ca" (OuterVolumeSpecName: "service-ca") pod "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" (UID: "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:35:02.479945 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479888 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" (UID: "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:35:02.479945 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.479923 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" (UID: "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:35:02.481626 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.481605 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" (UID: "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:35:02.481782 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.481761 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" (UID: "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:35:02.481895 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.481806 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-kube-api-access-cvhw8" (OuterVolumeSpecName: "kube-api-access-cvhw8") pod "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" (UID: "e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5"). InnerVolumeSpecName "kube-api-access-cvhw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:35:02.581137 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.581113 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-config\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:02.581137 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.581135 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-service-ca\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:02.581297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.581144 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvhw8\" (UniqueName: \"kubernetes.io/projected/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-kube-api-access-cvhw8\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:02.581297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.581153 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-oauth-config\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:02.581297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.581162 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-oauth-serving-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:02.581297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.581170 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-console-serving-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:02.581297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:02.581179 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5-trusted-ca-bundle\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:03.251562 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.251525 2573 generic.go:356] "Generic (PLEG): container finished" podID="e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" containerID="e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78" exitCode=0 Jul 08 19:35:03.252123 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.251582 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7897b8dd64-5d78v" Jul 08 19:35:03.252123 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.251606 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7897b8dd64-5d78v" event={"ID":"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5","Type":"ContainerDied","Data":"e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78"} Jul 08 19:35:03.252123 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.251659 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7897b8dd64-5d78v" event={"ID":"e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5","Type":"ContainerDied","Data":"4c44ee452248ed0c24010669da1f36dfb5c473b89bcd2b5f43c444e289e66be9"} Jul 08 19:35:03.252123 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.251673 2573 scope.go:117] "RemoveContainer" containerID="e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78" Jul 08 19:35:03.259997 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.259979 2573 scope.go:117] "RemoveContainer" containerID="e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78" Jul 08 19:35:03.260359 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:35:03.260341 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78\": container with ID starting with e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78 not found: ID does not exist" containerID="e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78" Jul 08 19:35:03.260410 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.260366 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78"} err="failed to get container status \"e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78\": rpc error: code = NotFound desc = could not find container \"e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78\": container with ID starting with e1bf6f72775a43335480bc5b0176bf6ac402271ac582bf1227ea4c58ffa33d78 not found: ID does not exist" Jul 08 19:35:03.268433 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.268411 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7897b8dd64-5d78v"] Jul 08 19:35:03.272451 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:03.272429 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7897b8dd64-5d78v"] Jul 08 19:35:04.567508 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:04.567474 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" path="/var/lib/kubelet/pods/e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5/volumes" Jul 08 19:35:06.210616 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:06.210586 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:35:06.211112 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:06.210656 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h92wd" Jul 08 19:35:24.115915 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.115877 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb"] Jul 08 19:35:24.116360 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.116104 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" podUID="8fb5d39b-5528-433f-ba49-5821c50e7a98" containerName="manager" containerID="cri-o://988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9" gracePeriod=10 Jul 08 19:35:24.357420 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.357391 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq"] Jul 08 19:35:24.357789 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.357774 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" containerName="console" Jul 08 19:35:24.357834 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.357791 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" containerName="console" Jul 08 19:35:24.357868 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.357860 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e16bc0ff-e5f9-487a-8aa4-a827ed7a82c5" containerName="console" Jul 08 19:35:24.358388 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.358370 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:35:24.359770 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.359751 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:24.362056 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.362012 2573 generic.go:356] "Generic (PLEG): container finished" podID="8fb5d39b-5528-433f-ba49-5821c50e7a98" containerID="988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9" exitCode=0 Jul 08 19:35:24.362056 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.362053 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" Jul 08 19:35:24.362183 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.362054 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" event={"ID":"8fb5d39b-5528-433f-ba49-5821c50e7a98","Type":"ContainerDied","Data":"988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9"} Jul 08 19:35:24.362183 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.362108 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb" event={"ID":"8fb5d39b-5528-433f-ba49-5821c50e7a98","Type":"ContainerDied","Data":"746f15738e5ea9c49fb29e2c0d228fe47bb850832c70ba68a921e1109f723a99"} Jul 08 19:35:24.362183 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.362133 2573 scope.go:117] "RemoveContainer" containerID="988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9" Jul 08 19:35:24.372804 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.372781 2573 scope.go:117] "RemoveContainer" containerID="988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9" Jul 08 19:35:24.373132 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:35:24.373110 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9\": container with ID starting with 988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9 not found: ID does not exist" containerID="988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9" Jul 08 19:35:24.373234 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.373141 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9"} err="failed to get container status \"988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9\": rpc error: code = NotFound desc = could not find container \"988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9\": container with ID starting with 988e14ce3c081376673b9d1c871d57b508351a9af9230d855a0a1ee5138a4fc9 not found: ID does not exist" Jul 08 19:35:24.377640 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.377610 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq"] Jul 08 19:35:24.475674 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.475635 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvl4m\" (UniqueName: \"kubernetes.io/projected/8fb5d39b-5528-433f-ba49-5821c50e7a98-kube-api-access-qvl4m\") pod \"8fb5d39b-5528-433f-ba49-5821c50e7a98\" (UID: \"8fb5d39b-5528-433f-ba49-5821c50e7a98\") " Jul 08 19:35:24.475862 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.475841 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fb5d39b-5528-433f-ba49-5821c50e7a98-extensions-socket-volume\") pod \"8fb5d39b-5528-433f-ba49-5821c50e7a98\" (UID: \"8fb5d39b-5528-433f-ba49-5821c50e7a98\") " Jul 08 19:35:24.476070 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.476009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv9gq\" (UniqueName: \"kubernetes.io/projected/a5111d37-26cc-40c7-b981-55640dfc8ff1-kube-api-access-fv9gq\") pod \"kuadrant-operator-controller-manager-86878c769f-9fvcq\" (UID: \"a5111d37-26cc-40c7-b981-55640dfc8ff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:24.476070 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.476069 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5111d37-26cc-40c7-b981-55640dfc8ff1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9fvcq\" (UID: \"a5111d37-26cc-40c7-b981-55640dfc8ff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:24.476262 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.476239 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fb5d39b-5528-433f-ba49-5821c50e7a98-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8fb5d39b-5528-433f-ba49-5821c50e7a98" (UID: "8fb5d39b-5528-433f-ba49-5821c50e7a98"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:35:24.477801 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.477778 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fb5d39b-5528-433f-ba49-5821c50e7a98-kube-api-access-qvl4m" (OuterVolumeSpecName: "kube-api-access-qvl4m") pod "8fb5d39b-5528-433f-ba49-5821c50e7a98" (UID: "8fb5d39b-5528-433f-ba49-5821c50e7a98"). InnerVolumeSpecName "kube-api-access-qvl4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:35:24.576653 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.576579 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv9gq\" (UniqueName: \"kubernetes.io/projected/a5111d37-26cc-40c7-b981-55640dfc8ff1-kube-api-access-fv9gq\") pod \"kuadrant-operator-controller-manager-86878c769f-9fvcq\" (UID: \"a5111d37-26cc-40c7-b981-55640dfc8ff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:24.576653 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.576619 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5111d37-26cc-40c7-b981-55640dfc8ff1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9fvcq\" (UID: \"a5111d37-26cc-40c7-b981-55640dfc8ff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:24.576797 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.576737 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fb5d39b-5528-433f-ba49-5821c50e7a98-extensions-socket-volume\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:24.576797 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.576753 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvl4m\" (UniqueName: \"kubernetes.io/projected/8fb5d39b-5528-433f-ba49-5821c50e7a98-kube-api-access-qvl4m\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:24.577153 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.577133 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5111d37-26cc-40c7-b981-55640dfc8ff1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9fvcq\" (UID: \"a5111d37-26cc-40c7-b981-55640dfc8ff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:24.584990 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.584964 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv9gq\" (UniqueName: \"kubernetes.io/projected/a5111d37-26cc-40c7-b981-55640dfc8ff1-kube-api-access-fv9gq\") pod \"kuadrant-operator-controller-manager-86878c769f-9fvcq\" (UID: \"a5111d37-26cc-40c7-b981-55640dfc8ff1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:24.671929 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.671855 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:24.679890 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.679855 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb"] Jul 08 19:35:24.686587 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.686560 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gfxjb"] Jul 08 19:35:24.809373 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:24.809347 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq"] Jul 08 19:35:24.811643 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:35:24.811614 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5111d37_26cc_40c7_b981_55640dfc8ff1.slice/crio-2da0beb9971862e1ca332f79aa3ad6e25ef254b1b395757b4f8b723765054571 WatchSource:0}: Error finding container 2da0beb9971862e1ca332f79aa3ad6e25ef254b1b395757b4f8b723765054571: Status 404 returned error can't find the container with id 2da0beb9971862e1ca332f79aa3ad6e25ef254b1b395757b4f8b723765054571 Jul 08 19:35:25.369242 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:25.369199 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" event={"ID":"a5111d37-26cc-40c7-b981-55640dfc8ff1","Type":"ContainerStarted","Data":"4fbe537655186e491c9c2d84627fa35432b05b77d71c3d99849a4b24c56f636b"} Jul 08 19:35:25.369690 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:25.369269 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" event={"ID":"a5111d37-26cc-40c7-b981-55640dfc8ff1","Type":"ContainerStarted","Data":"2da0beb9971862e1ca332f79aa3ad6e25ef254b1b395757b4f8b723765054571"} Jul 08 19:35:25.369690 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:25.369390 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:25.388110 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:25.388065 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" podStartSLOduration=1.3880474170000001 podStartE2EDuration="1.388047417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:35:25.387419962 +0000 UTC m=+687.044858585" watchObservedRunningTime="2026-07-08 19:35:25.388047417 +0000 UTC m=+687.045486016" Jul 08 19:35:26.567162 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:26.567133 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fb5d39b-5528-433f-ba49-5821c50e7a98" path="/var/lib/kubelet/pods/8fb5d39b-5528-433f-ba49-5821c50e7a98/volumes" Jul 08 19:35:36.378715 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:36.378683 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:35:40.577511 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.577476 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f"] Jul 08 19:35:40.577990 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.577974 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fb5d39b-5528-433f-ba49-5821c50e7a98" containerName="manager" Jul 08 19:35:40.578059 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.577994 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb5d39b-5528-433f-ba49-5821c50e7a98" containerName="manager" Jul 08 19:35:40.578104 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.578096 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fb5d39b-5528-433f-ba49-5821c50e7a98" containerName="manager" Jul 08 19:35:40.581311 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.581287 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.583687 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.583662 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-p6zdg\"" Jul 08 19:35:40.590848 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.590821 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f"] Jul 08 19:35:40.605352 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605321 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.605550 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.605550 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605411 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.605550 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605470 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.605550 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605507 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.605550 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605538 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.605810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605564 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.605810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605639 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9fv2\" (UniqueName: \"kubernetes.io/projected/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-kube-api-access-c9fv2\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.605810 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.605676 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706134 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706102 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706134 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706139 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706387 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9fv2\" (UniqueName: \"kubernetes.io/projected/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-kube-api-access-c9fv2\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706387 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706178 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706387 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706313 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706387 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706382 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706636 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706415 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706636 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706488 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706636 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706507 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706636 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706550 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706804 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706693 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.706933 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706909 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.707001 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.706915 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.707267 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.707243 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.708547 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.708518 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.708837 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.708816 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.715341 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.715314 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9fv2\" (UniqueName: \"kubernetes.io/projected/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-kube-api-access-c9fv2\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.715491 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.715475 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ed74d63-b86f-4bc3-a7ef-fe075ab472dd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-6rz4f\" (UID: \"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:40.893284 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:40.893190 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:41.032244 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:41.032216 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f"] Jul 08 19:35:41.034605 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:35:41.034567 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed74d63_b86f_4bc3_a7ef_fe075ab472dd.slice/crio-b61e2d4e75aeff39b0a64a1cd1a809788387be5951a60d69c9f6efdd6ddee1aa WatchSource:0}: Error finding container b61e2d4e75aeff39b0a64a1cd1a809788387be5951a60d69c9f6efdd6ddee1aa: Status 404 returned error can't find the container with id b61e2d4e75aeff39b0a64a1cd1a809788387be5951a60d69c9f6efdd6ddee1aa Jul 08 19:35:41.037225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:41.037194 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:35:41.037305 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:41.037265 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:35:41.037344 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:41.037308 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:35:41.456180 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:41.456132 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" event={"ID":"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd","Type":"ContainerStarted","Data":"22b6ed52baeaf5df2671f18fb92267238a27c08f4bf9149f54944b6fe4b0db26"} Jul 08 19:35:41.456367 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:41.456202 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" event={"ID":"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd","Type":"ContainerStarted","Data":"b61e2d4e75aeff39b0a64a1cd1a809788387be5951a60d69c9f6efdd6ddee1aa"} Jul 08 19:35:41.477299 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:41.477249 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" podStartSLOduration=1.47723257 podStartE2EDuration="1.47723257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:35:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:35:41.476183581 +0000 UTC m=+703.133622236" watchObservedRunningTime="2026-07-08 19:35:41.47723257 +0000 UTC m=+703.134671176" Jul 08 19:35:41.893749 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:41.893712 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:42.899041 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:42.898993 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:43.396236 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.396200 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hlqwl"] Jul 08 19:35:43.398994 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.398970 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hlqwl" Jul 08 19:35:43.401713 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.401692 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4v8tn\"" Jul 08 19:35:43.408662 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.408640 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlqwl"] Jul 08 19:35:43.426910 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.426870 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/855e3b86-adbf-49c5-94aa-3f77b9fb2a8f-kube-api-access-xjnjz\") pod \"authorino-58cd955675-hlqwl\" (UID: \"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f\") " pod="kuadrant-system/authorino-58cd955675-hlqwl" Jul 08 19:35:43.466640 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.466611 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:43.467601 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.467582 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:35:43.527973 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.527937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/855e3b86-adbf-49c5-94aa-3f77b9fb2a8f-kube-api-access-xjnjz\") pod \"authorino-58cd955675-hlqwl\" (UID: \"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f\") " pod="kuadrant-system/authorino-58cd955675-hlqwl" Jul 08 19:35:43.543436 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.543406 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/855e3b86-adbf-49c5-94aa-3f77b9fb2a8f-kube-api-access-xjnjz\") pod \"authorino-58cd955675-hlqwl\" (UID: \"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f\") " pod="kuadrant-system/authorino-58cd955675-hlqwl" Jul 08 19:35:43.710019 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.709943 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hlqwl" Jul 08 19:35:43.840768 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:43.840742 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlqwl"] Jul 08 19:35:43.843451 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:35:43.843427 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod855e3b86_adbf_49c5_94aa_3f77b9fb2a8f.slice/crio-d2d2520b2e71793ec825246db636b2daa97e2f65c0390a7d021e04d3169d4507 WatchSource:0}: Error finding container d2d2520b2e71793ec825246db636b2daa97e2f65c0390a7d021e04d3169d4507: Status 404 returned error can't find the container with id d2d2520b2e71793ec825246db636b2daa97e2f65c0390a7d021e04d3169d4507 Jul 08 19:35:44.474285 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:44.474246 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hlqwl" event={"ID":"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f","Type":"ContainerStarted","Data":"d2d2520b2e71793ec825246db636b2daa97e2f65c0390a7d021e04d3169d4507"} Jul 08 19:35:47.492996 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:47.492963 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hlqwl" event={"ID":"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f","Type":"ContainerStarted","Data":"f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472"} Jul 08 19:35:47.508901 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:47.508846 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hlqwl" podStartSLOduration=1.130643346 podStartE2EDuration="4.508831507s" totalImagesPullingTime="3.378188161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:35:47.507647473 +0000 UTC m=+709.165086092" watchObservedRunningTime="2026-07-08 19:35:47.508831507 +0000 UTC m=+709.166270113" Jul 08 19:35:49.285500 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:49.285463 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlqwl"] Jul 08 19:35:49.504175 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:49.504131 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hlqwl" podUID="855e3b86-adbf-49c5-94aa-3f77b9fb2a8f" containerName="authorino" containerID="cri-o://f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472" gracePeriod=30 Jul 08 19:35:49.754614 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:49.754590 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hlqwl" Jul 08 19:35:49.782889 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:49.782854 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/855e3b86-adbf-49c5-94aa-3f77b9fb2a8f-kube-api-access-xjnjz\") pod \"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f\" (UID: \"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f\") " Jul 08 19:35:49.784982 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:49.784952 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855e3b86-adbf-49c5-94aa-3f77b9fb2a8f-kube-api-access-xjnjz" (OuterVolumeSpecName: "kube-api-access-xjnjz") pod "855e3b86-adbf-49c5-94aa-3f77b9fb2a8f" (UID: "855e3b86-adbf-49c5-94aa-3f77b9fb2a8f"). InnerVolumeSpecName "kube-api-access-xjnjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:35:49.884171 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:49.884094 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjnjz\" (UniqueName: \"kubernetes.io/projected/855e3b86-adbf-49c5-94aa-3f77b9fb2a8f-kube-api-access-xjnjz\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:35:50.511621 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.511586 2573 generic.go:356] "Generic (PLEG): container finished" podID="855e3b86-adbf-49c5-94aa-3f77b9fb2a8f" containerID="f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472" exitCode=0 Jul 08 19:35:50.512084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.511630 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hlqwl" Jul 08 19:35:50.512084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.511673 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hlqwl" event={"ID":"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f","Type":"ContainerDied","Data":"f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472"} Jul 08 19:35:50.512084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.511716 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hlqwl" event={"ID":"855e3b86-adbf-49c5-94aa-3f77b9fb2a8f","Type":"ContainerDied","Data":"d2d2520b2e71793ec825246db636b2daa97e2f65c0390a7d021e04d3169d4507"} Jul 08 19:35:50.512084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.511733 2573 scope.go:117] "RemoveContainer" containerID="f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472" Jul 08 19:35:50.520842 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.520820 2573 scope.go:117] "RemoveContainer" containerID="f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472" Jul 08 19:35:50.521204 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:35:50.521178 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472\": container with ID starting with f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472 not found: ID does not exist" containerID="f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472" Jul 08 19:35:50.521281 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.521213 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472"} err="failed to get container status \"f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472\": rpc error: code = NotFound desc = could not find container \"f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472\": container with ID starting with f79e2f9365debb922bfd767e1094942a649d192b1f4d4ee911a6f0535c039472 not found: ID does not exist" Jul 08 19:35:50.533287 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.533263 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlqwl"] Jul 08 19:35:50.537494 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.537471 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlqwl"] Jul 08 19:35:50.567088 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:35:50.567061 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="855e3b86-adbf-49c5-94aa-3f77b9fb2a8f" path="/var/lib/kubelet/pods/855e3b86-adbf-49c5-94aa-3f77b9fb2a8f/volumes" Jul 08 19:36:10.925891 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:10.925845 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5l98j"] Jul 08 19:36:10.926742 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:10.926714 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855e3b86-adbf-49c5-94aa-3f77b9fb2a8f" containerName="authorino" Jul 08 19:36:10.926866 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:10.926746 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="855e3b86-adbf-49c5-94aa-3f77b9fb2a8f" containerName="authorino" Jul 08 19:36:10.930833 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:10.926923 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="855e3b86-adbf-49c5-94aa-3f77b9fb2a8f" containerName="authorino" Jul 08 19:36:10.936665 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:10.936626 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5l98j"] Jul 08 19:36:10.936803 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:10.936733 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5l98j" Jul 08 19:36:10.939201 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:10.939178 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4v8tn\"" Jul 08 19:36:11.070846 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.070812 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmx9r\" (UniqueName: \"kubernetes.io/projected/3eb9b14a-d62f-49fe-be4c-1e99e471dd8e-kube-api-access-qmx9r\") pod \"authorino-5d4b67c446-5l98j\" (UID: \"3eb9b14a-d62f-49fe-be4c-1e99e471dd8e\") " pod="kuadrant-system/authorino-5d4b67c446-5l98j" Jul 08 19:36:11.111780 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.111741 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5l98j"] Jul 08 19:36:11.112018 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:36:11.112001 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qmx9r], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-5l98j" podUID="3eb9b14a-d62f-49fe-be4c-1e99e471dd8e" Jul 08 19:36:11.137313 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.137280 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-qfwqw"] Jul 08 19:36:11.139557 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.139532 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.141903 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.141885 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 19:36:11.150201 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.150178 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-qfwqw"] Jul 08 19:36:11.171679 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.171639 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmx9r\" (UniqueName: \"kubernetes.io/projected/3eb9b14a-d62f-49fe-be4c-1e99e471dd8e-kube-api-access-qmx9r\") pod \"authorino-5d4b67c446-5l98j\" (UID: \"3eb9b14a-d62f-49fe-be4c-1e99e471dd8e\") " pod="kuadrant-system/authorino-5d4b67c446-5l98j" Jul 08 19:36:11.182970 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.182898 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmx9r\" (UniqueName: \"kubernetes.io/projected/3eb9b14a-d62f-49fe-be4c-1e99e471dd8e-kube-api-access-qmx9r\") pod \"authorino-5d4b67c446-5l98j\" (UID: \"3eb9b14a-d62f-49fe-be4c-1e99e471dd8e\") " pod="kuadrant-system/authorino-5d4b67c446-5l98j" Jul 08 19:36:11.212531 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.212501 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-qfwqw"] Jul 08 19:36:11.212782 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:36:11.212762 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mgbkz tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-mgbkz tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-qfwqw" podUID="89d7669a-25f9-473c-8346-fb6a8d751230" Jul 08 19:36:11.242857 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.242823 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c9db8c4b7-5clb7"] Jul 08 19:36:11.245257 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.245233 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:36:11.253994 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.253957 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c9db8c4b7-5clb7"] Jul 08 19:36:11.272373 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.272329 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgbkz\" (UniqueName: \"kubernetes.io/projected/89d7669a-25f9-473c-8346-fb6a8d751230-kube-api-access-mgbkz\") pod \"authorino-76877994-qfwqw\" (UID: \"89d7669a-25f9-473c-8346-fb6a8d751230\") " pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.272533 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.272430 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89d7669a-25f9-473c-8346-fb6a8d751230-tls-cert\") pod \"authorino-76877994-qfwqw\" (UID: \"89d7669a-25f9-473c-8346-fb6a8d751230\") " pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.372901 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.372859 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89d7669a-25f9-473c-8346-fb6a8d751230-tls-cert\") pod \"authorino-76877994-qfwqw\" (UID: \"89d7669a-25f9-473c-8346-fb6a8d751230\") " pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.373109 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.372917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqj6k\" (UniqueName: \"kubernetes.io/projected/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-kube-api-access-tqj6k\") pod \"authorino-6c9db8c4b7-5clb7\" (UID: \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\") " pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:36:11.373109 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.372949 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-tls-cert\") pod \"authorino-6c9db8c4b7-5clb7\" (UID: \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\") " pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:36:11.373109 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.372970 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgbkz\" (UniqueName: \"kubernetes.io/projected/89d7669a-25f9-473c-8346-fb6a8d751230-kube-api-access-mgbkz\") pod \"authorino-76877994-qfwqw\" (UID: \"89d7669a-25f9-473c-8346-fb6a8d751230\") " pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.375355 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.375337 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89d7669a-25f9-473c-8346-fb6a8d751230-tls-cert\") pod \"authorino-76877994-qfwqw\" (UID: \"89d7669a-25f9-473c-8346-fb6a8d751230\") " pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.381141 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.381102 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgbkz\" (UniqueName: \"kubernetes.io/projected/89d7669a-25f9-473c-8346-fb6a8d751230-kube-api-access-mgbkz\") pod \"authorino-76877994-qfwqw\" (UID: \"89d7669a-25f9-473c-8346-fb6a8d751230\") " pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.473905 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.473813 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqj6k\" (UniqueName: \"kubernetes.io/projected/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-kube-api-access-tqj6k\") pod \"authorino-6c9db8c4b7-5clb7\" (UID: \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\") " pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:36:11.473905 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.473867 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-tls-cert\") pod \"authorino-6c9db8c4b7-5clb7\" (UID: \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\") " pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:36:11.476308 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.476287 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-tls-cert\") pod \"authorino-6c9db8c4b7-5clb7\" (UID: \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\") " pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:36:11.482174 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.482152 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqj6k\" (UniqueName: \"kubernetes.io/projected/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-kube-api-access-tqj6k\") pod \"authorino-6c9db8c4b7-5clb7\" (UID: \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\") " pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:36:11.555128 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.555088 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:36:11.627983 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.627954 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5l98j" Jul 08 19:36:11.628164 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.627954 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.635361 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.635334 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5l98j" Jul 08 19:36:11.638305 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.638283 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:11.694528 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.694505 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c9db8c4b7-5clb7"] Jul 08 19:36:11.696903 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:36:11.696874 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0b3e99a_0fd2_4650_b2a0_6c552f9e4c0a.slice/crio-38a8c4add03a792bbc448dfd449b5d8b7ae8c927c79b50ce3f7a83cf94766660 WatchSource:0}: Error finding container 38a8c4add03a792bbc448dfd449b5d8b7ae8c927c79b50ce3f7a83cf94766660: Status 404 returned error can't find the container with id 38a8c4add03a792bbc448dfd449b5d8b7ae8c927c79b50ce3f7a83cf94766660 Jul 08 19:36:11.776730 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.776691 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmx9r\" (UniqueName: \"kubernetes.io/projected/3eb9b14a-d62f-49fe-be4c-1e99e471dd8e-kube-api-access-qmx9r\") pod \"3eb9b14a-d62f-49fe-be4c-1e99e471dd8e\" (UID: \"3eb9b14a-d62f-49fe-be4c-1e99e471dd8e\") " Jul 08 19:36:11.776906 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.776753 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89d7669a-25f9-473c-8346-fb6a8d751230-tls-cert\") pod \"89d7669a-25f9-473c-8346-fb6a8d751230\" (UID: \"89d7669a-25f9-473c-8346-fb6a8d751230\") " Jul 08 19:36:11.776906 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.776789 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgbkz\" (UniqueName: \"kubernetes.io/projected/89d7669a-25f9-473c-8346-fb6a8d751230-kube-api-access-mgbkz\") pod \"89d7669a-25f9-473c-8346-fb6a8d751230\" (UID: \"89d7669a-25f9-473c-8346-fb6a8d751230\") " Jul 08 19:36:11.778956 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.778921 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89d7669a-25f9-473c-8346-fb6a8d751230-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "89d7669a-25f9-473c-8346-fb6a8d751230" (UID: "89d7669a-25f9-473c-8346-fb6a8d751230"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:36:11.779086 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.778984 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eb9b14a-d62f-49fe-be4c-1e99e471dd8e-kube-api-access-qmx9r" (OuterVolumeSpecName: "kube-api-access-qmx9r") pod "3eb9b14a-d62f-49fe-be4c-1e99e471dd8e" (UID: "3eb9b14a-d62f-49fe-be4c-1e99e471dd8e"). InnerVolumeSpecName "kube-api-access-qmx9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:36:11.779086 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.778996 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89d7669a-25f9-473c-8346-fb6a8d751230-kube-api-access-mgbkz" (OuterVolumeSpecName: "kube-api-access-mgbkz") pod "89d7669a-25f9-473c-8346-fb6a8d751230" (UID: "89d7669a-25f9-473c-8346-fb6a8d751230"). InnerVolumeSpecName "kube-api-access-mgbkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:36:11.877583 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.877555 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmx9r\" (UniqueName: \"kubernetes.io/projected/3eb9b14a-d62f-49fe-be4c-1e99e471dd8e-kube-api-access-qmx9r\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:36:11.877583 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.877577 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/89d7669a-25f9-473c-8346-fb6a8d751230-tls-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:36:11.877583 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:11.877587 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgbkz\" (UniqueName: \"kubernetes.io/projected/89d7669a-25f9-473c-8346-fb6a8d751230-kube-api-access-mgbkz\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:36:12.634262 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.634219 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" event={"ID":"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a","Type":"ContainerStarted","Data":"dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948"} Jul 08 19:36:12.634262 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.634242 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-5l98j" Jul 08 19:36:12.634262 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.634265 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" event={"ID":"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a","Type":"ContainerStarted","Data":"38a8c4add03a792bbc448dfd449b5d8b7ae8c927c79b50ce3f7a83cf94766660"} Jul 08 19:36:12.634796 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.634403 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-qfwqw" Jul 08 19:36:12.675628 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.675562 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" podStartSLOduration=1.172406666 podStartE2EDuration="1.675541704s" totalImagesPullingTime="503.135038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:36:12.650653767 +0000 UTC m=+734.308092374" watchObservedRunningTime="2026-07-08 19:36:12.675541704 +0000 UTC m=+734.332980311" Jul 08 19:36:12.685707 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.685673 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-qfwqw"] Jul 08 19:36:12.690376 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.690332 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-qfwqw"] Jul 08 19:36:12.712530 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.712491 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5l98j"] Jul 08 19:36:12.716582 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:12.716552 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-5l98j"] Jul 08 19:36:14.572755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:14.569790 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eb9b14a-d62f-49fe-be4c-1e99e471dd8e" path="/var/lib/kubelet/pods/3eb9b14a-d62f-49fe-be4c-1e99e471dd8e/volumes" Jul 08 19:36:14.572755 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:14.570290 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89d7669a-25f9-473c-8346-fb6a8d751230" path="/var/lib/kubelet/pods/89d7669a-25f9-473c-8346-fb6a8d751230/volumes" Jul 08 19:36:59.507233 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:59.507189 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6rz4f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 19:36:59.507691 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:36:59.507286 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" podUID="0ed74d63-b86f-4bc3-a7ef-fe075ab472dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 19:37:00.895330 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:00.895233 2573 generic.go:356] "Generic (PLEG): container finished" podID="0ed74d63-b86f-4bc3-a7ef-fe075ab472dd" containerID="22b6ed52baeaf5df2671f18fb92267238a27c08f4bf9149f54944b6fe4b0db26" exitCode=0 Jul 08 19:37:00.895330 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:00.895315 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" event={"ID":"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd","Type":"ContainerDied","Data":"22b6ed52baeaf5df2671f18fb92267238a27c08f4bf9149f54944b6fe4b0db26"} Jul 08 19:37:00.895799 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:00.895779 2573 scope.go:117] "RemoveContainer" containerID="22b6ed52baeaf5df2671f18fb92267238a27c08f4bf9149f54944b6fe4b0db26" Jul 08 19:37:00.896798 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:00.896771 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:37:00.896857 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:00.896843 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:37:00.896894 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:00.896879 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:37:01.902066 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:01.902018 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" event={"ID":"0ed74d63-b86f-4bc3-a7ef-fe075ab472dd","Type":"ContainerStarted","Data":"272c10d240822c023cb6255fa12dc5e001fb20205a7f3cb12532a445f34d4543"} Jul 08 19:37:02.893915 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:02.893878 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:37:02.894168 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:02.894147 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6rz4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 08 19:37:02.894218 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:02.894193 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" podUID="0ed74d63-b86f-4bc3-a7ef-fe075ab472dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 08 19:37:03.894425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:03.894386 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6rz4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 08 19:37:03.894853 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:03.894442 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" podUID="0ed74d63-b86f-4bc3-a7ef-fe075ab472dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 08 19:37:04.894221 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:04.894181 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6rz4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 08 19:37:04.894429 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:04.894243 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" podUID="0ed74d63-b86f-4bc3-a7ef-fe075ab472dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 08 19:37:05.894417 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:05.894379 2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-6rz4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 08 19:37:05.894928 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:05.894455 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" podUID="0ed74d63-b86f-4bc3-a7ef-fe075ab472dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 08 19:37:06.897529 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:06.897492 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:37:06.897914 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:06.897770 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:37:06.898385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:06.898367 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-6rz4f" Jul 08 19:37:29.075131 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.075087 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds"] Jul 08 19:37:29.079167 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.079138 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.081701 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.081657 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 19:37:29.082760 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.082556 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 19:37:29.082760 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.082621 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 08 19:37:29.082968 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.082803 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m5fhc\"" Jul 08 19:37:29.086225 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.086061 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds"] Jul 08 19:37:29.141702 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.141670 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.141881 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.141716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.141881 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.141737 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.141881 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.141754 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.141881 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.141770 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f58668d1-15c8-42a4-a316-ca66cc7d367b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.141881 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.141793 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrl6f\" (UniqueName: \"kubernetes.io/projected/f58668d1-15c8-42a4-a316-ca66cc7d367b-kube-api-access-lrl6f\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.242333 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.242295 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrl6f\" (UniqueName: \"kubernetes.io/projected/f58668d1-15c8-42a4-a316-ca66cc7d367b-kube-api-access-lrl6f\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.242513 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.242396 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.242513 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.242445 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.242513 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.242469 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.242685 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.242598 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.242685 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.242640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f58668d1-15c8-42a4-a316-ca66cc7d367b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.243052 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.242995 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.243169 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.243057 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.243169 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.243056 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.245134 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.245101 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f58668d1-15c8-42a4-a316-ca66cc7d367b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.245642 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.245619 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f58668d1-15c8-42a4-a316-ca66cc7d367b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.250789 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.250759 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrl6f\" (UniqueName: \"kubernetes.io/projected/f58668d1-15c8-42a4-a316-ca66cc7d367b-kube-api-access-lrl6f\") pod \"e2e-distinct-simulated-kserve-59975d69f9-2h8ds\" (UID: \"f58668d1-15c8-42a4-a316-ca66cc7d367b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.393262 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.393173 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:29.533464 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:29.533425 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds"] Jul 08 19:37:29.536656 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:37:29.536624 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58668d1_15c8_42a4_a316_ca66cc7d367b.slice/crio-27142dd696ec0db4b3466731ddf8e034c332639726d0d5c7964a8fcc3693b6e7 WatchSource:0}: Error finding container 27142dd696ec0db4b3466731ddf8e034c332639726d0d5c7964a8fcc3693b6e7: Status 404 returned error can't find the container with id 27142dd696ec0db4b3466731ddf8e034c332639726d0d5c7964a8fcc3693b6e7 Jul 08 19:37:30.052358 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:30.052324 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" event={"ID":"f58668d1-15c8-42a4-a316-ca66cc7d367b","Type":"ContainerStarted","Data":"27142dd696ec0db4b3466731ddf8e034c332639726d0d5c7964a8fcc3693b6e7"} Jul 08 19:37:31.059827 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:31.059737 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" event={"ID":"f58668d1-15c8-42a4-a316-ca66cc7d367b","Type":"ContainerStarted","Data":"da4f8778c815727ff7174a01fdcbc9acbd77edc872a905a62968e99f86248e32"} Jul 08 19:37:31.079188 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:31.079135 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" podStartSLOduration=0.864847275 podStartE2EDuration="2.079116944s" totalImagesPullingTime="1.214269669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:37:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:37:31.077409095 +0000 UTC m=+812.734847701" watchObservedRunningTime="2026-07-08 19:37:31.079116944 +0000 UTC m=+812.736555549" Jul 08 19:37:39.393795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:39.393757 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:39.394289 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:39.393821 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:39.406350 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:39.406320 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:40.117946 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:40.117919 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-2h8ds" Jul 08 19:37:43.190926 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.190892 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4"] Jul 08 19:37:43.196904 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.196883 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.199519 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.199496 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 08 19:37:43.205813 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.205789 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4"] Jul 08 19:37:43.255872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.255835 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.256097 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.255898 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.256097 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.255922 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/612c66e9-602f-446a-9639-d13c7122e332-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.256097 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.255962 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.256283 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.256102 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgln2\" (UniqueName: \"kubernetes.io/projected/612c66e9-602f-446a-9639-d13c7122e332-kube-api-access-zgln2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.256283 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.256190 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357103 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357279 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357128 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgln2\" (UniqueName: \"kubernetes.io/projected/612c66e9-602f-446a-9639-d13c7122e332-kube-api-access-zgln2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357279 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357161 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357279 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357187 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357279 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357222 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357502 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357351 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/612c66e9-602f-446a-9639-d13c7122e332-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357502 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357482 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357502 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357493 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.357625 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.357564 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.362332 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.362301 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/612c66e9-602f-446a-9639-d13c7122e332-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.362596 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.362572 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/612c66e9-602f-446a-9639-d13c7122e332-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.367813 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.367779 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgln2\" (UniqueName: \"kubernetes.io/projected/612c66e9-602f-446a-9639-d13c7122e332-kube-api-access-zgln2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4\" (UID: \"612c66e9-602f-446a-9639-d13c7122e332\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.508047 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.507999 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:43.646504 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:43.646476 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4"] Jul 08 19:37:43.648631 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:37:43.648595 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612c66e9_602f_446a_9639_d13c7122e332.slice/crio-c23d680cee5925019b9aec4a86f953507a5c0c41d10196914697de72878321f3 WatchSource:0}: Error finding container c23d680cee5925019b9aec4a86f953507a5c0c41d10196914697de72878321f3: Status 404 returned error can't find the container with id c23d680cee5925019b9aec4a86f953507a5c0c41d10196914697de72878321f3 Jul 08 19:37:44.131546 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:44.131455 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" event={"ID":"612c66e9-602f-446a-9639-d13c7122e332","Type":"ContainerStarted","Data":"dc7d3309404e0a611cd0dd978a87181ac92048605b35936ecc73892b15e3c716"} Jul 08 19:37:44.131546 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:44.131508 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" event={"ID":"612c66e9-602f-446a-9639-d13c7122e332","Type":"ContainerStarted","Data":"c23d680cee5925019b9aec4a86f953507a5c0c41d10196914697de72878321f3"} Jul 08 19:37:44.149007 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:44.148947 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" podStartSLOduration=0.954634398 podStartE2EDuration="1.148930039s" totalImagesPullingTime="194.295641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:37:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:37:44.14735902 +0000 UTC m=+825.804797627" watchObservedRunningTime="2026-07-08 19:37:44.148930039 +0000 UTC m=+825.806368645" Jul 08 19:37:53.508685 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:53.508637 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:53.509235 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:53.508720 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:53.521756 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:53.521723 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:37:54.197404 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:37:54.197366 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4" Jul 08 19:38:56.688734 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.688655 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-9dfdcffcd-2dmmp"] Jul 08 19:38:56.692075 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.692050 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" Jul 08 19:38:56.700425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.700398 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9dfdcffcd-2dmmp"] Jul 08 19:38:56.808596 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.808556 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c059129-08de-4e6f-8b2a-47a598ec4a33-tls-cert\") pod \"authorino-9dfdcffcd-2dmmp\" (UID: \"7c059129-08de-4e6f-8b2a-47a598ec4a33\") " pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" Jul 08 19:38:56.808773 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.808638 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxl52\" (UniqueName: \"kubernetes.io/projected/7c059129-08de-4e6f-8b2a-47a598ec4a33-kube-api-access-mxl52\") pod \"authorino-9dfdcffcd-2dmmp\" (UID: \"7c059129-08de-4e6f-8b2a-47a598ec4a33\") " pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" Jul 08 19:38:56.909998 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.909962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxl52\" (UniqueName: \"kubernetes.io/projected/7c059129-08de-4e6f-8b2a-47a598ec4a33-kube-api-access-mxl52\") pod \"authorino-9dfdcffcd-2dmmp\" (UID: \"7c059129-08de-4e6f-8b2a-47a598ec4a33\") " pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" Jul 08 19:38:56.910193 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.910053 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c059129-08de-4e6f-8b2a-47a598ec4a33-tls-cert\") pod \"authorino-9dfdcffcd-2dmmp\" (UID: \"7c059129-08de-4e6f-8b2a-47a598ec4a33\") " pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" Jul 08 19:38:56.912538 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.912514 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7c059129-08de-4e6f-8b2a-47a598ec4a33-tls-cert\") pod \"authorino-9dfdcffcd-2dmmp\" (UID: \"7c059129-08de-4e6f-8b2a-47a598ec4a33\") " pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" Jul 08 19:38:56.917873 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:56.917852 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxl52\" (UniqueName: \"kubernetes.io/projected/7c059129-08de-4e6f-8b2a-47a598ec4a33-kube-api-access-mxl52\") pod \"authorino-9dfdcffcd-2dmmp\" (UID: \"7c059129-08de-4e6f-8b2a-47a598ec4a33\") " pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" Jul 08 19:38:57.002743 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:57.002710 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" Jul 08 19:38:57.140738 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:57.140712 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-9dfdcffcd-2dmmp"] Jul 08 19:38:57.142719 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:38:57.142690 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c059129_08de_4e6f_8b2a_47a598ec4a33.slice/crio-cdb20f1753e87a71e30d73ae087552074921746e7a68e27484bb8460cd64fad9 WatchSource:0}: Error finding container cdb20f1753e87a71e30d73ae087552074921746e7a68e27484bb8460cd64fad9: Status 404 returned error can't find the container with id cdb20f1753e87a71e30d73ae087552074921746e7a68e27484bb8460cd64fad9 Jul 08 19:38:57.537824 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:57.537785 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" event={"ID":"7c059129-08de-4e6f-8b2a-47a598ec4a33","Type":"ContainerStarted","Data":"cdb20f1753e87a71e30d73ae087552074921746e7a68e27484bb8460cd64fad9"} Jul 08 19:38:58.545350 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:58.545314 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" event={"ID":"7c059129-08de-4e6f-8b2a-47a598ec4a33","Type":"ContainerStarted","Data":"6c41339b859c9be8c7a2f8a43b028de27880f16d78708c15fe4e3f7364ffe11b"} Jul 08 19:38:58.575055 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:58.574982 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-9dfdcffcd-2dmmp" podStartSLOduration=2.128629935 podStartE2EDuration="2.574964987s" totalImagesPullingTime="446.335052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:38:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:38:58.572753014 +0000 UTC m=+900.230191621" watchObservedRunningTime="2026-07-08 19:38:58.574964987 +0000 UTC m=+900.232403617" Jul 08 19:38:58.640506 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:58.640467 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c9db8c4b7-5clb7"] Jul 08 19:38:58.640993 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:58.640959 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" podUID="d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a" containerName="authorino" containerID="cri-o://dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948" gracePeriod=30 Jul 08 19:38:58.916322 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:58.916291 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:38:59.033286 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.033251 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-tls-cert\") pod \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\" (UID: \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\") " Jul 08 19:38:59.033497 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.033368 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqj6k\" (UniqueName: \"kubernetes.io/projected/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-kube-api-access-tqj6k\") pod \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\" (UID: \"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a\") " Jul 08 19:38:59.035983 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.035952 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-kube-api-access-tqj6k" (OuterVolumeSpecName: "kube-api-access-tqj6k") pod "d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a" (UID: "d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a"). InnerVolumeSpecName "kube-api-access-tqj6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:38:59.044589 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.044558 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a" (UID: "d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:38:59.134289 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.134198 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-tls-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:38:59.134289 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.134232 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqj6k\" (UniqueName: \"kubernetes.io/projected/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a-kube-api-access-tqj6k\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:38:59.553038 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.552987 2573 generic.go:356] "Generic (PLEG): container finished" podID="d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a" containerID="dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948" exitCode=0 Jul 08 19:38:59.553495 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.553057 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" Jul 08 19:38:59.553495 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.553067 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" event={"ID":"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a","Type":"ContainerDied","Data":"dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948"} Jul 08 19:38:59.553495 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.553142 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9db8c4b7-5clb7" event={"ID":"d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a","Type":"ContainerDied","Data":"38a8c4add03a792bbc448dfd449b5d8b7ae8c927c79b50ce3f7a83cf94766660"} Jul 08 19:38:59.553495 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.553172 2573 scope.go:117] "RemoveContainer" containerID="dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948" Jul 08 19:38:59.563366 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.563338 2573 scope.go:117] "RemoveContainer" containerID="dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948" Jul 08 19:38:59.563726 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:38:59.563699 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948\": container with ID starting with dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948 not found: ID does not exist" containerID="dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948" Jul 08 19:38:59.563851 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.563738 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948"} err="failed to get container status \"dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948\": rpc error: code = NotFound desc = could not find container \"dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948\": container with ID starting with dffd450037defb3f093a82af415fdfe1503c6cf74e6bc2b80961774ac894e948 not found: ID does not exist" Jul 08 19:38:59.632158 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.632113 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c9db8c4b7-5clb7"] Jul 08 19:38:59.634202 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:38:59.634178 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c9db8c4b7-5clb7"] Jul 08 19:39:00.567245 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:39:00.567211 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a" path="/var/lib/kubelet/pods/d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a/volumes" Jul 08 19:41:01.562699 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:41:01.562664 2573 scope.go:117] "RemoveContainer" containerID="05c8690a351ff53aad1f72253584752eb2d2cc54d81d8692c44af943e5a17565" Jul 08 19:45:03.389871 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:03.389774 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq"] Jul 08 19:45:03.390490 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:03.390040 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" podUID="a5111d37-26cc-40c7-b981-55640dfc8ff1" containerName="manager" containerID="cri-o://4fbe537655186e491c9c2d84627fa35432b05b77d71c3d99849a4b24c56f636b" gracePeriod=10 Jul 08 19:45:06.559717 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:06.559686 2573 generic.go:356] "Generic (PLEG): container finished" podID="a5111d37-26cc-40c7-b981-55640dfc8ff1" containerID="4fbe537655186e491c9c2d84627fa35432b05b77d71c3d99849a4b24c56f636b" exitCode=0 Jul 08 19:45:06.560106 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:06.559761 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" event={"ID":"a5111d37-26cc-40c7-b981-55640dfc8ff1","Type":"ContainerDied","Data":"4fbe537655186e491c9c2d84627fa35432b05b77d71c3d99849a4b24c56f636b"} Jul 08 19:45:07.145780 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.145755 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:45:07.260897 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.260804 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5111d37-26cc-40c7-b981-55640dfc8ff1-extensions-socket-volume\") pod \"a5111d37-26cc-40c7-b981-55640dfc8ff1\" (UID: \"a5111d37-26cc-40c7-b981-55640dfc8ff1\") " Jul 08 19:45:07.260897 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.260892 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv9gq\" (UniqueName: \"kubernetes.io/projected/a5111d37-26cc-40c7-b981-55640dfc8ff1-kube-api-access-fv9gq\") pod \"a5111d37-26cc-40c7-b981-55640dfc8ff1\" (UID: \"a5111d37-26cc-40c7-b981-55640dfc8ff1\") " Jul 08 19:45:07.261294 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.261260 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5111d37-26cc-40c7-b981-55640dfc8ff1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a5111d37-26cc-40c7-b981-55640dfc8ff1" (UID: "a5111d37-26cc-40c7-b981-55640dfc8ff1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:45:07.262983 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.262960 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5111d37-26cc-40c7-b981-55640dfc8ff1-kube-api-access-fv9gq" (OuterVolumeSpecName: "kube-api-access-fv9gq") pod "a5111d37-26cc-40c7-b981-55640dfc8ff1" (UID: "a5111d37-26cc-40c7-b981-55640dfc8ff1"). InnerVolumeSpecName "kube-api-access-fv9gq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:45:07.361530 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.361494 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv9gq\" (UniqueName: \"kubernetes.io/projected/a5111d37-26cc-40c7-b981-55640dfc8ff1-kube-api-access-fv9gq\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:45:07.361530 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.361524 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5111d37-26cc-40c7-b981-55640dfc8ff1-extensions-socket-volume\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:45:07.566528 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.566442 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" Jul 08 19:45:07.566528 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.566456 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq" event={"ID":"a5111d37-26cc-40c7-b981-55640dfc8ff1","Type":"ContainerDied","Data":"2da0beb9971862e1ca332f79aa3ad6e25ef254b1b395757b4f8b723765054571"} Jul 08 19:45:07.566528 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.566508 2573 scope.go:117] "RemoveContainer" containerID="4fbe537655186e491c9c2d84627fa35432b05b77d71c3d99849a4b24c56f636b" Jul 08 19:45:07.590305 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.590277 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq"] Jul 08 19:45:07.594469 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:07.594437 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9fvcq"] Jul 08 19:45:08.568327 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:45:08.568294 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5111d37-26cc-40c7-b981-55640dfc8ff1" path="/var/lib/kubelet/pods/a5111d37-26cc-40c7-b981-55640dfc8ff1/volumes" Jul 08 19:46:04.577938 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.577905 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt"] Jul 08 19:46:04.578385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.578269 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5111d37-26cc-40c7-b981-55640dfc8ff1" containerName="manager" Jul 08 19:46:04.578385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.578281 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5111d37-26cc-40c7-b981-55640dfc8ff1" containerName="manager" Jul 08 19:46:04.578385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.578294 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a" containerName="authorino" Jul 08 19:46:04.578385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.578299 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a" containerName="authorino" Jul 08 19:46:04.578385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.578358 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5111d37-26cc-40c7-b981-55640dfc8ff1" containerName="manager" Jul 08 19:46:04.578385 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.578367 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b3e99a-0fd2-4650-b2a0-6c552f9e4c0a" containerName="authorino" Jul 08 19:46:04.582676 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.582654 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:04.585259 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.585238 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jx8qd\"" Jul 08 19:46:04.593749 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.593721 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt"] Jul 08 19:46:04.678189 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.678153 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b75d7dc-7b27-408f-a9c3-1d968c071597-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gtkqt\" (UID: \"6b75d7dc-7b27-408f-a9c3-1d968c071597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:04.678365 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.678199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52nr\" (UniqueName: \"kubernetes.io/projected/6b75d7dc-7b27-408f-a9c3-1d968c071597-kube-api-access-b52nr\") pod \"kuadrant-operator-controller-manager-86878c769f-gtkqt\" (UID: \"6b75d7dc-7b27-408f-a9c3-1d968c071597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:04.779686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.779645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b75d7dc-7b27-408f-a9c3-1d968c071597-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gtkqt\" (UID: \"6b75d7dc-7b27-408f-a9c3-1d968c071597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:04.779686 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.779692 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52nr\" (UniqueName: \"kubernetes.io/projected/6b75d7dc-7b27-408f-a9c3-1d968c071597-kube-api-access-b52nr\") pod \"kuadrant-operator-controller-manager-86878c769f-gtkqt\" (UID: \"6b75d7dc-7b27-408f-a9c3-1d968c071597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:04.780107 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.780084 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b75d7dc-7b27-408f-a9c3-1d968c071597-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gtkqt\" (UID: \"6b75d7dc-7b27-408f-a9c3-1d968c071597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:04.788396 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.788365 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52nr\" (UniqueName: \"kubernetes.io/projected/6b75d7dc-7b27-408f-a9c3-1d968c071597-kube-api-access-b52nr\") pod \"kuadrant-operator-controller-manager-86878c769f-gtkqt\" (UID: \"6b75d7dc-7b27-408f-a9c3-1d968c071597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:04.894389 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:04.894296 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:05.040724 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:05.040691 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt"] Jul 08 19:46:05.043624 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:46:05.043594 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b75d7dc_7b27_408f_a9c3_1d968c071597.slice/crio-ba3cc1b350b9f78e48e51660ef1343e2b4a8944a91b740e9cccd966b9ac21556 WatchSource:0}: Error finding container ba3cc1b350b9f78e48e51660ef1343e2b4a8944a91b740e9cccd966b9ac21556: Status 404 returned error can't find the container with id ba3cc1b350b9f78e48e51660ef1343e2b4a8944a91b740e9cccd966b9ac21556 Jul 08 19:46:05.876835 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:05.876795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" event={"ID":"6b75d7dc-7b27-408f-a9c3-1d968c071597","Type":"ContainerStarted","Data":"6abf84680569beaa354ffe27ab3c0cc229520b94b9d9b325387056e5964fdd9e"} Jul 08 19:46:05.877321 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:05.876849 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" event={"ID":"6b75d7dc-7b27-408f-a9c3-1d968c071597","Type":"ContainerStarted","Data":"ba3cc1b350b9f78e48e51660ef1343e2b4a8944a91b740e9cccd966b9ac21556"} Jul 08 19:46:05.877321 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:05.876889 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:46:05.897298 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:05.897246 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" podStartSLOduration=1.897229692 podStartE2EDuration="1.897229692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:46:05.896505391 +0000 UTC m=+1327.553944038" watchObservedRunningTime="2026-07-08 19:46:05.897229692 +0000 UTC m=+1327.554668299" Jul 08 19:46:16.885817 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:46:16.885781 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gtkqt" Jul 08 19:52:36.767445 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.767364 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx"] Jul 08 19:52:36.771194 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.771168 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.774478 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.774453 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5a9c03f1-openshift-default-dockercfg-ccfmd\"" Jul 08 19:52:36.800892 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.800858 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx"] Jul 08 19:52:36.855571 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855537 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-socket\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.855571 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855575 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680b860a-9f17-4288-9079-31cf5022206c-istiod-ca-cert\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.855783 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855601 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-envoy\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.855783 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855634 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-certs\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.855783 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855652 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mbqx\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-kube-api-access-5mbqx\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.855783 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855715 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-data\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.855783 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855739 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680b860a-9f17-4288-9079-31cf5022206c-istio-podinfo\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.855783 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855753 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-istio-token\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.855981 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.855825 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-credential-socket\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957175 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957138 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-data\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957175 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957182 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680b860a-9f17-4288-9079-31cf5022206c-istio-podinfo\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957208 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-istio-token\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957257 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-credential-socket\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957450 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957308 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-socket\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957449 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680b860a-9f17-4288-9079-31cf5022206c-istiod-ca-cert\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957500 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-envoy\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957542 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-certs\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957569 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mbqx\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-kube-api-access-5mbqx\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957619 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957608 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-data\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957678 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-credential-socket\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957688 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-socket\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.957980 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.957927 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-certs\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.958199 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.958178 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680b860a-9f17-4288-9079-31cf5022206c-istiod-ca-cert\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.959730 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.959708 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-envoy\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.959826 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.959759 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680b860a-9f17-4288-9079-31cf5022206c-istio-podinfo\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.968101 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.968072 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mbqx\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-kube-api-access-5mbqx\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:36.968211 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:36.968144 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-istio-token\") pod \"e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:37.082728 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:37.082630 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:37.223788 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:37.223760 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx"] Jul 08 19:52:37.226242 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:52:37.226200 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680b860a_9f17_4288_9079_31cf5022206c.slice/crio-02671f4e80585cd761a4191e8035e8c1f45921419bd550e1a44be2b986dfaf2f WatchSource:0}: Error finding container 02671f4e80585cd761a4191e8035e8c1f45921419bd550e1a44be2b986dfaf2f: Status 404 returned error can't find the container with id 02671f4e80585cd761a4191e8035e8c1f45921419bd550e1a44be2b986dfaf2f Jul 08 19:52:37.229016 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:37.228977 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:52:37.229135 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:37.229087 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:52:37.229182 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:37.229133 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:52:38.028328 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:38.028276 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" event={"ID":"680b860a-9f17-4288-9079-31cf5022206c","Type":"ContainerStarted","Data":"d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0"} Jul 08 19:52:38.028836 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:38.028354 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" event={"ID":"680b860a-9f17-4288-9079-31cf5022206c","Type":"ContainerStarted","Data":"02671f4e80585cd761a4191e8035e8c1f45921419bd550e1a44be2b986dfaf2f"} Jul 08 19:52:38.057254 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:38.057188 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" podStartSLOduration=2.057168049 podStartE2EDuration="2.057168049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:38.0560868 +0000 UTC m=+1719.713525407" watchObservedRunningTime="2026-07-08 19:52:38.057168049 +0000 UTC m=+1719.714606656" Jul 08 19:52:38.083669 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:38.083637 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:38.088405 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:38.088378 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:39.034208 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:39.034179 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:39.035164 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:39.035143 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:43.241702 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:43.241656 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx"] Jul 08 19:52:43.242663 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:43.242623 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" podUID="680b860a-9f17-4288-9079-31cf5022206c" containerName="istio-proxy" containerID="cri-o://d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0" gracePeriod=30 Jul 08 19:52:48.893442 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.893408 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:48.978846 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.978734 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-socket\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.978846 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.978787 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-credential-socket\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.978846 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.978817 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-istio-token\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.979166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.978863 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680b860a-9f17-4288-9079-31cf5022206c-istiod-ca-cert\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.979166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.978889 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-certs\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.979166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.978912 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680b860a-9f17-4288-9079-31cf5022206c-istio-podinfo\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.979166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.978952 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-envoy\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.979166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979003 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-data\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.979166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979063 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mbqx\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-kube-api-access-5mbqx\") pod \"680b860a-9f17-4288-9079-31cf5022206c\" (UID: \"680b860a-9f17-4288-9079-31cf5022206c\") " Jul 08 19:52:48.979166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979081 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:52:48.979166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979127 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:52:48.979549 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979230 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:52:48.979549 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979428 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:48.979549 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979448 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:48.979549 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979455 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680b860a-9f17-4288-9079-31cf5022206c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:52:48.979549 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.979461 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:48.980148 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.980113 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-data" (OuterVolumeSpecName: "istio-data") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:52:48.981242 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.981209 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:52:48.981347 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.981245 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/680b860a-9f17-4288-9079-31cf5022206c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:52:48.981347 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.981305 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-istio-token" (OuterVolumeSpecName: "istio-token") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:52:48.981504 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:48.981485 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-kube-api-access-5mbqx" (OuterVolumeSpecName: "kube-api-access-5mbqx") pod "680b860a-9f17-4288-9079-31cf5022206c" (UID: "680b860a-9f17-4288-9079-31cf5022206c"). InnerVolumeSpecName "kube-api-access-5mbqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:52:49.080102 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.080070 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:49.080102 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.080099 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mbqx\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-kube-api-access-5mbqx\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:49.080102 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.080108 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/680b860a-9f17-4288-9079-31cf5022206c-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:49.080348 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.080117 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/680b860a-9f17-4288-9079-31cf5022206c-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:49.080348 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.080125 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/680b860a-9f17-4288-9079-31cf5022206c-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:49.080348 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.080132 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/680b860a-9f17-4288-9079-31cf5022206c-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:52:49.104115 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.104078 2573 generic.go:356] "Generic (PLEG): container finished" podID="680b860a-9f17-4288-9079-31cf5022206c" containerID="d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0" exitCode=0 Jul 08 19:52:49.104309 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.104132 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" Jul 08 19:52:49.104309 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.104163 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" event={"ID":"680b860a-9f17-4288-9079-31cf5022206c","Type":"ContainerDied","Data":"d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0"} Jul 08 19:52:49.104309 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.104244 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx" event={"ID":"680b860a-9f17-4288-9079-31cf5022206c","Type":"ContainerDied","Data":"02671f4e80585cd761a4191e8035e8c1f45921419bd550e1a44be2b986dfaf2f"} Jul 08 19:52:49.104309 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.104274 2573 scope.go:117] "RemoveContainer" containerID="d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0" Jul 08 19:52:49.116274 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.116252 2573 scope.go:117] "RemoveContainer" containerID="d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0" Jul 08 19:52:49.116581 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:52:49.116559 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0\": container with ID starting with d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0 not found: ID does not exist" containerID="d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0" Jul 08 19:52:49.116630 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.116591 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0"} err="failed to get container status \"d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0\": rpc error: code = NotFound desc = could not find container \"d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0\": container with ID starting with d918bfb941b1ce02c9cf842e878fe94700b14b52f770d14cf453ccb61b32fdd0 not found: ID does not exist" Jul 08 19:52:49.131675 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.131641 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx"] Jul 08 19:52:49.135772 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.135742 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5a9c03f1-openshift-default-547b944b59-krxxx"] Jul 08 19:52:49.583092 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.583056 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp"] Jul 08 19:52:49.583581 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.583556 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="680b860a-9f17-4288-9079-31cf5022206c" containerName="istio-proxy" Jul 08 19:52:49.583581 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.583580 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="680b860a-9f17-4288-9079-31cf5022206c" containerName="istio-proxy" Jul 08 19:52:49.583708 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.583694 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="680b860a-9f17-4288-9079-31cf5022206c" containerName="istio-proxy" Jul 08 19:52:49.594200 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.594165 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.596845 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.596814 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8e698564-openshift-default-dockercfg-xwkqm\"" Jul 08 19:52:49.599707 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.599627 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp"] Jul 08 19:52:49.685420 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685383 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-envoy\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.685420 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685426 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-credential-socket\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.685693 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685489 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-data\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.685693 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685541 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-socket\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.685693 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685574 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-token\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.685693 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685604 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-certs\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.685693 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzxt5\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-kube-api-access-qzxt5\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.685959 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-podinfo\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.685959 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.685740 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bab0d2d2-364d-491d-a375-8166367ac1b3-istiod-ca-cert\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787133 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-envoy\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787171 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-credential-socket\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787416 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-data\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787416 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787224 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-socket\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787416 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787245 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-token\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787416 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787325 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-certs\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787416 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787367 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzxt5\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-kube-api-access-qzxt5\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787416 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787414 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-podinfo\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787717 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787437 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bab0d2d2-364d-491d-a375-8166367ac1b3-istiod-ca-cert\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787717 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787648 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-socket\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787808 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787716 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-credential-socket\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787808 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787774 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-data\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.787905 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.787888 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-certs\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.788146 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.788127 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bab0d2d2-364d-491d-a375-8166367ac1b3-istiod-ca-cert\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.789481 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.789460 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-envoy\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.789821 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.789805 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-podinfo\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.795369 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.795348 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzxt5\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-kube-api-access-qzxt5\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.795722 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.795700 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-token\") pod \"e2e-ait-8e698564-openshift-default-684c87c645-lqnzp\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:49.907920 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:49.907827 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:50.048827 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:50.048799 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp"] Jul 08 19:52:50.050171 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:52:50.050146 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab0d2d2_364d_491d_a375_8166367ac1b3.slice/crio-e7ae9388d09abfbbbdb571c14a7df1a2e6a43db1ba4562a090ba6cbedd03d462 WatchSource:0}: Error finding container e7ae9388d09abfbbbdb571c14a7df1a2e6a43db1ba4562a090ba6cbedd03d462: Status 404 returned error can't find the container with id e7ae9388d09abfbbbdb571c14a7df1a2e6a43db1ba4562a090ba6cbedd03d462 Jul 08 19:52:50.052423 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:50.052384 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:52:50.052490 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:50.052476 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:52:50.052528 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:50.052516 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:52:50.112270 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:50.112243 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" event={"ID":"bab0d2d2-364d-491d-a375-8166367ac1b3","Type":"ContainerStarted","Data":"e7ae9388d09abfbbbdb571c14a7df1a2e6a43db1ba4562a090ba6cbedd03d462"} Jul 08 19:52:50.567313 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:50.567284 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="680b860a-9f17-4288-9079-31cf5022206c" path="/var/lib/kubelet/pods/680b860a-9f17-4288-9079-31cf5022206c/volumes" Jul 08 19:52:51.118667 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:51.118630 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" event={"ID":"bab0d2d2-364d-491d-a375-8166367ac1b3","Type":"ContainerStarted","Data":"9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81"} Jul 08 19:52:51.142666 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:51.142603 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" podStartSLOduration=2.142584465 podStartE2EDuration="2.142584465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:52:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:52:51.139605277 +0000 UTC m=+1732.797043883" watchObservedRunningTime="2026-07-08 19:52:51.142584465 +0000 UTC m=+1732.800023070" Jul 08 19:52:51.908273 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:51.908235 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:51.912629 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:51.912607 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:52.124655 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:52.124626 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:52:52.125671 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:52:52.125652 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:53:01.187394 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:01.187355 2573 generic.go:356] "Generic (PLEG): container finished" podID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerID="9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81" exitCode=0 Jul 08 19:53:01.187840 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:01.187433 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" event={"ID":"bab0d2d2-364d-491d-a375-8166367ac1b3","Type":"ContainerDied","Data":"9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81"} Jul 08 19:53:01.187906 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:01.187891 2573 scope.go:117] "RemoveContainer" containerID="9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81" Jul 08 19:53:01.188970 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:01.188937 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:53:01.189107 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:01.189038 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:53:01.189107 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:01.189090 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:53:02.195034 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:02.194986 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" event={"ID":"bab0d2d2-364d-491d-a375-8166367ac1b3","Type":"ContainerStarted","Data":"caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d"} Jul 08 19:53:02.281722 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:02.281689 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp"] Jul 08 19:53:02.908946 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:02.908900 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:53:02.913884 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:02.913849 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:53:03.201071 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:03.200969 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" podUID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerName="istio-proxy" containerID="cri-o://caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d" gracePeriod=30 Jul 08 19:53:03.201553 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:03.201125 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:53:03.202077 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:03.202061 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:53:08.458435 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.458411 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:53:08.570904 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.570845 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-credential-socket\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.571379 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.571129 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzxt5\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-kube-api-access-qzxt5\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.571379 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.571172 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-podinfo\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.571379 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.571307 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-token\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.571993 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.571448 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-data\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.571993 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.571479 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-socket\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.571993 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.571637 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-certs\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.571993 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.571924 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bab0d2d2-364d-491d-a375-8166367ac1b3-istiod-ca-cert\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.572627 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.572044 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-envoy\") pod \"bab0d2d2-364d-491d-a375-8166367ac1b3\" (UID: \"bab0d2d2-364d-491d-a375-8166367ac1b3\") " Jul 08 19:53:08.574650 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.574610 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:08.574874 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.574836 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:08.575079 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.574848 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bab0d2d2-364d-491d-a375-8166367ac1b3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:53:08.575170 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.574982 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:08.575518 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.575490 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:08.575636 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.575616 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-data" (OuterVolumeSpecName: "istio-data") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:08.577059 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.577000 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:53:08.577059 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.577018 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-kube-api-access-qzxt5" (OuterVolumeSpecName: "kube-api-access-qzxt5") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "kube-api-access-qzxt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:53:08.577195 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.577174 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-token" (OuterVolumeSpecName: "istio-token") pod "bab0d2d2-364d-491d-a375-8166367ac1b3" (UID: "bab0d2d2-364d-491d-a375-8166367ac1b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:53:08.603716 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.603677 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j"] Jul 08 19:53:08.604082 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.604068 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerName="istio-proxy" Jul 08 19:53:08.604141 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.604083 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerName="istio-proxy" Jul 08 19:53:08.604180 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.604162 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerName="istio-proxy" Jul 08 19:53:08.607365 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.607341 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.609820 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.609795 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-1d4d50e0-openshift-default-dockercfg-4mpgb\"" Jul 08 19:53:08.619546 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.619515 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j"] Jul 08 19:53:08.674264 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674233 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bab0d2d2-364d-491d-a375-8166367ac1b3-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.674264 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674260 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.674264 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674271 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.674516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674281 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzxt5\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-kube-api-access-qzxt5\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.674516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674291 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.674516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674300 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.674516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674307 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.674516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674314 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.674516 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.674321 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bab0d2d2-364d-491d-a375-8166367ac1b3-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:08.775849 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.775814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-credential-socket\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.776036 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.775868 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf95d\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-kube-api-access-jf95d\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.776036 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.775901 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-token\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.776036 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.775967 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istiod-ca-cert\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.776036 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.776004 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-socket\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.776190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.776111 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-data\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.776190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.776137 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-podinfo\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.776190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.776165 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-certs\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.776190 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.776178 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-envoy\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877197 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877101 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-certs\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877197 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877144 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-envoy\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877197 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877189 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-credential-socket\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877468 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877213 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf95d\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-kube-api-access-jf95d\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877468 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877232 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-token\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877468 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istiod-ca-cert\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877468 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877342 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-socket\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877468 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877380 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-data\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877468 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-podinfo\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877792 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877554 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-certs\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877792 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877739 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-socket\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877792 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877746 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-data\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.877961 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.877841 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-credential-socket\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.878150 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.878126 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istiod-ca-cert\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.879649 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.879627 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-envoy\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.879811 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.879794 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-podinfo\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.885417 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.885384 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf95d\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-kube-api-access-jf95d\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.885565 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.885548 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-token\") pod \"e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:08.918747 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:08.918713 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:09.061872 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.061845 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j"] Jul 08 19:53:09.063382 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:53:09.063358 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4884086_0eb1_4e61_84f8_4c2fedf79cb1.slice/crio-6748d91d0e1a2bee91a8689fbe8b621ed2b534437aac0b5af32e0ffab39a7f44 WatchSource:0}: Error finding container 6748d91d0e1a2bee91a8689fbe8b621ed2b534437aac0b5af32e0ffab39a7f44: Status 404 returned error can't find the container with id 6748d91d0e1a2bee91a8689fbe8b621ed2b534437aac0b5af32e0ffab39a7f44 Jul 08 19:53:09.065474 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.065449 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:53:09.065547 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.065510 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:53:09.065547 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.065542 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:53:09.234774 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.234732 2573 generic.go:356] "Generic (PLEG): container finished" podID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerID="caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d" exitCode=0 Jul 08 19:53:09.234970 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.234813 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" Jul 08 19:53:09.234970 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.234826 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" event={"ID":"bab0d2d2-364d-491d-a375-8166367ac1b3","Type":"ContainerDied","Data":"caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d"} Jul 08 19:53:09.234970 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.234899 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp" event={"ID":"bab0d2d2-364d-491d-a375-8166367ac1b3","Type":"ContainerDied","Data":"e7ae9388d09abfbbbdb571c14a7df1a2e6a43db1ba4562a090ba6cbedd03d462"} Jul 08 19:53:09.234970 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.234934 2573 scope.go:117] "RemoveContainer" containerID="caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d" Jul 08 19:53:09.237049 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.236997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" event={"ID":"c4884086-0eb1-4e61-84f8-4c2fedf79cb1","Type":"ContainerStarted","Data":"934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b"} Jul 08 19:53:09.237380 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.237345 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" event={"ID":"c4884086-0eb1-4e61-84f8-4c2fedf79cb1","Type":"ContainerStarted","Data":"6748d91d0e1a2bee91a8689fbe8b621ed2b534437aac0b5af32e0ffab39a7f44"} Jul 08 19:53:09.245927 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.245904 2573 scope.go:117] "RemoveContainer" containerID="9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81" Jul 08 19:53:09.255109 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.255088 2573 scope.go:117] "RemoveContainer" containerID="caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d" Jul 08 19:53:09.255425 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:53:09.255400 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d\": container with ID starting with caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d not found: ID does not exist" containerID="caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d" Jul 08 19:53:09.255487 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.255439 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d"} err="failed to get container status \"caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d\": rpc error: code = NotFound desc = could not find container \"caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d\": container with ID starting with caab7f2a6120fb96aca1da9f92db834ec0e50181549459375c80699806bb000d not found: ID does not exist" Jul 08 19:53:09.255487 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.255468 2573 scope.go:117] "RemoveContainer" containerID="9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81" Jul 08 19:53:09.255758 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:53:09.255740 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81\": container with ID starting with 9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81 not found: ID does not exist" containerID="9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81" Jul 08 19:53:09.255804 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.255766 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81"} err="failed to get container status \"9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81\": rpc error: code = NotFound desc = could not find container \"9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81\": container with ID starting with 9f55cb08626bd5977950ceb6aee2bfd9e1fffad5b8e43302ede6495303d71f81 not found: ID does not exist" Jul 08 19:53:09.261966 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.261917 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" podStartSLOduration=1.261901277 podStartE2EDuration="1.261901277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:53:09.259814287 +0000 UTC m=+1750.917252925" watchObservedRunningTime="2026-07-08 19:53:09.261901277 +0000 UTC m=+1750.919339966" Jul 08 19:53:09.275722 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.275690 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp"] Jul 08 19:53:09.279575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.279548 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8e698564-openshift-default-684c87c645-lqnzp"] Jul 08 19:53:09.919796 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:09.919764 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:10.568555 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:10.568522 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bab0d2d2-364d-491d-a375-8166367ac1b3" path="/var/lib/kubelet/pods/bab0d2d2-364d-491d-a375-8166367ac1b3/volumes" Jul 08 19:53:10.925575 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:10.925489 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:11.249163 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:11.249127 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:11.250299 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:11.250279 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:14.353741 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:14.353679 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j"] Jul 08 19:53:14.354720 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:14.354676 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" podUID="c4884086-0eb1-4e61-84f8-4c2fedf79cb1" containerName="istio-proxy" containerID="cri-o://934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b" gracePeriod=30 Jul 08 19:53:21.613474 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.613407 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:21.698397 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698360 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-data\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698577 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698416 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-credential-socket\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698577 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698438 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-podinfo\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698577 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698454 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-token\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698577 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698470 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-socket\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698577 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698541 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istiod-ca-cert\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698577 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698568 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf95d\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-kube-api-access-jf95d\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698905 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698617 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-certs\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698905 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698644 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-envoy\") pod \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\" (UID: \"c4884086-0eb1-4e61-84f8-4c2fedf79cb1\") " Jul 08 19:53:21.698905 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698755 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:21.699106 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.698946 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:21.699297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.699165 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:21.699297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.699195 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:21.699297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.699197 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:21.699297 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.699246 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:53:21.699564 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.699373 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-data" (OuterVolumeSpecName: "istio-data") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:21.700760 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.700736 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:53:21.700858 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.700778 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:53:21.700858 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.700796 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:53:21.701072 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.701046 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-kube-api-access-jf95d" (OuterVolumeSpecName: "kube-api-access-jf95d") pod "c4884086-0eb1-4e61-84f8-4c2fedf79cb1" (UID: "c4884086-0eb1-4e61-84f8-4c2fedf79cb1"). InnerVolumeSpecName "kube-api-access-jf95d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:53:21.800034 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.799995 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:21.800207 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.800066 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf95d\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-kube-api-access-jf95d\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:21.800207 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.800077 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:21.800207 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.800086 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:21.800207 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.800094 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:21.800207 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.800102 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:21.800207 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:21.800109 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4884086-0eb1-4e61-84f8-4c2fedf79cb1-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:53:22.307762 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.307727 2573 generic.go:356] "Generic (PLEG): container finished" podID="c4884086-0eb1-4e61-84f8-4c2fedf79cb1" containerID="934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b" exitCode=0 Jul 08 19:53:22.307974 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.307781 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" Jul 08 19:53:22.307974 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.307810 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" event={"ID":"c4884086-0eb1-4e61-84f8-4c2fedf79cb1","Type":"ContainerDied","Data":"934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b"} Jul 08 19:53:22.307974 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.307892 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j" event={"ID":"c4884086-0eb1-4e61-84f8-4c2fedf79cb1","Type":"ContainerDied","Data":"6748d91d0e1a2bee91a8689fbe8b621ed2b534437aac0b5af32e0ffab39a7f44"} Jul 08 19:53:22.307974 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.307915 2573 scope.go:117] "RemoveContainer" containerID="934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b" Jul 08 19:53:22.317430 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.317411 2573 scope.go:117] "RemoveContainer" containerID="934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b" Jul 08 19:53:22.317693 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:53:22.317673 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b\": container with ID starting with 934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b not found: ID does not exist" containerID="934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b" Jul 08 19:53:22.317744 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.317703 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b"} err="failed to get container status \"934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b\": rpc error: code = NotFound desc = could not find container \"934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b\": container with ID starting with 934ff666ae02abfc5ef0fb4ab4226279dfa8a472f554bcf1d70154daee0a928b not found: ID does not exist" Jul 08 19:53:22.333944 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.333911 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j"] Jul 08 19:53:22.337746 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.337720 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-1d4d50e0-openshift-default-5f7c5c9dd7-v288j"] Jul 08 19:53:22.568176 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:53:22.568095 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4884086-0eb1-4e61-84f8-4c2fedf79cb1" path="/var/lib/kubelet/pods/c4884086-0eb1-4e61-84f8-4c2fedf79cb1/volumes" Jul 08 19:54:56.744098 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.744065 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw"] Jul 08 19:54:56.744584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.744419 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerName="istio-proxy" Jul 08 19:54:56.744584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.744429 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerName="istio-proxy" Jul 08 19:54:56.744584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.744448 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4884086-0eb1-4e61-84f8-4c2fedf79cb1" containerName="istio-proxy" Jul 08 19:54:56.744584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.744454 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4884086-0eb1-4e61-84f8-4c2fedf79cb1" containerName="istio-proxy" Jul 08 19:54:56.744584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.744514 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bab0d2d2-364d-491d-a375-8166367ac1b3" containerName="istio-proxy" Jul 08 19:54:56.744584 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.744522 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4884086-0eb1-4e61-84f8-4c2fedf79cb1" containerName="istio-proxy" Jul 08 19:54:56.747682 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.747665 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.750435 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.750412 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e9d7beea-openshift-default-dockercfg-vnk6x\"" Jul 08 19:54:56.767916 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.767890 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw"] Jul 08 19:54:56.945090 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4rjs\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-kube-api-access-x4rjs\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.945292 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-credential-socket\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.945292 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945128 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istiod-ca-cert\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.945292 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945154 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-certs\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.945292 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945270 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-envoy\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.945494 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945324 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-data\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.945494 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945407 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-podinfo\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.945494 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945439 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-socket\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:56.945494 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:56.945458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-token\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046083 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.045958 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4rjs\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-kube-api-access-x4rjs\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046083 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046009 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-credential-socket\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046083 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046051 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istiod-ca-cert\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046380 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046132 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-certs\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046380 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046295 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-envoy\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046380 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046352 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-data\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046555 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046387 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-podinfo\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046555 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046414 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-socket\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046555 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046421 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-credential-socket\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046555 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046440 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-token\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046555 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046461 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-certs\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046836 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046758 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-data\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046928 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046907 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istiod-ca-cert\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.046994 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.046949 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-socket\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.048471 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.048448 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-envoy\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.048837 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.048816 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-podinfo\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.053859 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.053836 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-token\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.053859 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.053855 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4rjs\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-kube-api-access-x4rjs\") pod \"e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.057911 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.057891 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:57.207873 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.207849 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw"] Jul 08 19:54:57.209648 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:54:57.209617 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b6c0bba_dd1a_4595_9c28_f99ca1c9c935.slice/crio-2de1535d73de5bd0eb86b636b82ad2da49431f0a757128d5c844a595157fb4bc WatchSource:0}: Error finding container 2de1535d73de5bd0eb86b636b82ad2da49431f0a757128d5c844a595157fb4bc: Status 404 returned error can't find the container with id 2de1535d73de5bd0eb86b636b82ad2da49431f0a757128d5c844a595157fb4bc Jul 08 19:54:57.211823 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.211783 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:54:57.211902 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.211883 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:54:57.211941 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.211930 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:54:57.826663 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.826621 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" event={"ID":"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935","Type":"ContainerStarted","Data":"5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969"} Jul 08 19:54:57.827138 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.826685 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" event={"ID":"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935","Type":"ContainerStarted","Data":"2de1535d73de5bd0eb86b636b82ad2da49431f0a757128d5c844a595157fb4bc"} Jul 08 19:54:57.849320 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:57.849266 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" podStartSLOduration=1.849249628 podStartE2EDuration="1.849249628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:54:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:54:57.847107035 +0000 UTC m=+1859.504545654" watchObservedRunningTime="2026-07-08 19:54:57.849249628 +0000 UTC m=+1859.506688234" Jul 08 19:54:58.058359 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:58.058329 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:58.063332 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:58.063308 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:58.832811 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:58.832781 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:54:58.833774 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:54:58.833755 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:55:13.834578 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:13.834539 2573 patch_prober.go:28] interesting pod/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 08 19:55:13.835014 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:13.834627 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 08 19:55:13.919758 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:13.919722 2573 generic.go:356] "Generic (PLEG): container finished" podID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerID="5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969" exitCode=0 Jul 08 19:55:13.919924 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:13.919795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" event={"ID":"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935","Type":"ContainerDied","Data":"5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969"} Jul 08 19:55:13.920291 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:13.920274 2573 scope.go:117] "RemoveContainer" containerID="5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969" Jul 08 19:55:13.921285 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:13.921258 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:13.921345 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:13.921328 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:13.921381 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:13.921362 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:14.927080 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:14.927019 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" event={"ID":"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935","Type":"ContainerStarted","Data":"cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f"} Jul 08 19:55:15.059107 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:15.059073 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:55:15.063107 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:15.063082 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:55:15.933715 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:15.933686 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:55:15.934706 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:15.934675 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:55:18.326647 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:18.326613 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw"] Jul 08 19:55:18.950074 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:18.949996 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerName="istio-proxy" containerID="cri-o://cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f" gracePeriod=30 Jul 08 19:55:24.209793 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.209765 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:55:24.293998 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.293968 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-podinfo\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294235 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294018 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-data\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294235 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294107 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-token\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294235 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294222 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-certs\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294261 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-credential-socket\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294288 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istiod-ca-cert\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294365 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4rjs\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-kube-api-access-x4rjs\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294396 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-socket\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294481 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:55:24.294795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294496 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:55:24.294795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294505 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-envoy\") pod \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\" (UID: \"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935\") " Jul 08 19:55:24.294795 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294732 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:55:24.295357 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294826 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-data" (OuterVolumeSpecName: "istio-data") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:55:24.295357 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294877 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.295357 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294899 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.295357 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294914 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.295357 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.294947 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:55:24.296493 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.296461 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-token" (OuterVolumeSpecName: "istio-token") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:55:24.296597 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.296531 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-kube-api-access-x4rjs" (OuterVolumeSpecName: "kube-api-access-x4rjs") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "kube-api-access-x4rjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:55:24.296651 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.296632 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:55:24.296715 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.296696 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" (UID: "0b6c0bba-dd1a-4595-9c28-f99ca1c9c935"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:55:24.396386 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.396327 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.396386 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.396378 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4rjs\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-kube-api-access-x4rjs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.396386 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.396392 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.396641 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.396404 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.396641 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.396416 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.396641 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.396427 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:55:24.982167 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.982071 2573 generic.go:356] "Generic (PLEG): container finished" podID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerID="cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f" exitCode=0 Jul 08 19:55:24.982167 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.982134 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" Jul 08 19:55:24.982167 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.982153 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" event={"ID":"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935","Type":"ContainerDied","Data":"cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f"} Jul 08 19:55:24.982411 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.982213 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw" event={"ID":"0b6c0bba-dd1a-4595-9c28-f99ca1c9c935","Type":"ContainerDied","Data":"2de1535d73de5bd0eb86b636b82ad2da49431f0a757128d5c844a595157fb4bc"} Jul 08 19:55:24.982411 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.982244 2573 scope.go:117] "RemoveContainer" containerID="cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f" Jul 08 19:55:24.991438 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:24.991418 2573 scope.go:117] "RemoveContainer" containerID="5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969" Jul 08 19:55:25.000624 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:25.000605 2573 scope.go:117] "RemoveContainer" containerID="cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f" Jul 08 19:55:25.001013 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:55:25.000982 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f\": container with ID starting with cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f not found: ID does not exist" containerID="cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f" Jul 08 19:55:25.001165 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:25.001047 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f"} err="failed to get container status \"cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f\": rpc error: code = NotFound desc = could not find container \"cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f\": container with ID starting with cb9520782b7ccdb1a04da5efa32378b2459433085391d0d19a6a82913ebfc68f not found: ID does not exist" Jul 08 19:55:25.001165 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:25.001081 2573 scope.go:117] "RemoveContainer" containerID="5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969" Jul 08 19:55:25.001165 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:25.001152 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw"] Jul 08 19:55:25.001487 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:55:25.001459 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969\": container with ID starting with 5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969 not found: ID does not exist" containerID="5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969" Jul 08 19:55:25.001610 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:25.001490 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969"} err="failed to get container status \"5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969\": rpc error: code = NotFound desc = could not find container \"5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969\": container with ID starting with 5c62c9c0d46376de6c747042e4f2e14d3d0c49c9d94c967ebdf2444ec463d969 not found: ID does not exist" Jul 08 19:55:25.004285 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:25.004264 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e9d7beea-openshift-default-79f8d97bff-lfqvw"] Jul 08 19:55:26.567890 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:26.567856 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" path="/var/lib/kubelet/pods/0b6c0bba-dd1a-4595-9c28-f99ca1c9c935/volumes" Jul 08 19:55:38.443063 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.443005 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm"] Jul 08 19:55:38.443537 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.443443 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerName="istio-proxy" Jul 08 19:55:38.443537 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.443456 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerName="istio-proxy" Jul 08 19:55:38.443537 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.443475 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerName="istio-proxy" Jul 08 19:55:38.443537 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.443482 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerName="istio-proxy" Jul 08 19:55:38.443672 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.443547 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerName="istio-proxy" Jul 08 19:55:38.445678 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.445658 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.448185 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.448161 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c4f984b0-openshift-default-dockercfg-th9wc\"" Jul 08 19:55:38.457196 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.457160 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm"] Jul 08 19:55:38.532929 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.532874 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-data\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.533158 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.532969 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-token\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.533158 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.533011 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-certs\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.533158 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.533128 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-podinfo\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.533339 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.533175 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-credential-socket\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.533339 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.533225 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx48t\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-kube-api-access-nx48t\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.533339 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.533285 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-socket\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.533529 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.533362 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-envoy\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.533529 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.533397 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istiod-ca-cert\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.633987 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.633945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istiod-ca-cert\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.633987 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.633983 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-data\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634035 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-token\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634074 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-certs\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-podinfo\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634129 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-credential-socket\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634157 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx48t\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-kube-api-access-nx48t\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634204 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-socket\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634276 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-envoy\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634643 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634524 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-data\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634643 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634595 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-certs\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634851 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634828 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-credential-socket\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634919 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634863 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-socket\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.634919 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.634903 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istiod-ca-cert\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.636436 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.636411 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-envoy\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.636787 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.636770 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-podinfo\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.644080 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.644015 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx48t\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-kube-api-access-nx48t\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.644219 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.644200 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-token\") pod \"e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.769837 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.769807 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:38.936564 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.936533 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm"] Jul 08 19:55:38.938976 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:55:38.938940 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod620b6f0a_aab7_4cad_9012_8b1cce13dc6f.slice/crio-b9e915092ae1260b5b9f3ffd0164205e0300ede46c267f8ab53efe5ed4608d36 WatchSource:0}: Error finding container b9e915092ae1260b5b9f3ffd0164205e0300ede46c267f8ab53efe5ed4608d36: Status 404 returned error can't find the container with id b9e915092ae1260b5b9f3ffd0164205e0300ede46c267f8ab53efe5ed4608d36 Jul 08 19:55:38.941782 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.941733 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:38.941893 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.941827 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:38.941893 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:38.941856 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:39.073278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:39.072315 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" event={"ID":"620b6f0a-aab7-4cad-9012-8b1cce13dc6f","Type":"ContainerStarted","Data":"ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0"} Jul 08 19:55:39.073278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:39.072385 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" event={"ID":"620b6f0a-aab7-4cad-9012-8b1cce13dc6f","Type":"ContainerStarted","Data":"b9e915092ae1260b5b9f3ffd0164205e0300ede46c267f8ab53efe5ed4608d36"} Jul 08 19:55:39.095765 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:39.095699 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" podStartSLOduration=1.095675173 podStartE2EDuration="1.095675173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:55:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:55:39.093905044 +0000 UTC m=+1900.751343652" watchObservedRunningTime="2026-07-08 19:55:39.095675173 +0000 UTC m=+1900.753113782" Jul 08 19:55:39.770035 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:39.769994 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:39.774930 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:39.774899 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:40.077681 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:40.077589 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:40.078695 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:40.078665 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:52.151905 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:52.151867 2573 generic.go:356] "Generic (PLEG): container finished" podID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerID="ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0" exitCode=0 Jul 08 19:55:52.152357 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:52.151946 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" event={"ID":"620b6f0a-aab7-4cad-9012-8b1cce13dc6f","Type":"ContainerDied","Data":"ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0"} Jul 08 19:55:52.152481 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:52.152465 2573 scope.go:117] "RemoveContainer" containerID="ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0" Jul 08 19:55:52.153532 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:52.153506 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:52.153637 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:52.153586 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:52.153707 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:52.153643 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:55:53.160845 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:53.160806 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" event={"ID":"620b6f0a-aab7-4cad-9012-8b1cce13dc6f","Type":"ContainerStarted","Data":"7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42"} Jul 08 19:55:53.769995 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:53.769952 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:53.771061 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:53.771017 2573 patch_prober.go:28] interesting pod/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 08 19:55:53.771179 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:53.771088 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 08 19:55:54.770367 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:54.770328 2573 patch_prober.go:28] interesting pod/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 08 19:55:54.770794 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:54.770385 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 08 19:55:55.077732 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:55.077644 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:55.770509 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:55.770468 2573 patch_prober.go:28] interesting pod/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 08 19:55:55.770929 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:55.770530 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 08 19:55:56.770569 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:56.770532 2573 patch_prober.go:28] interesting pod/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 08 19:55:56.770992 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:56.770587 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 08 19:55:57.774854 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:57.774822 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:57.775860 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:57.775839 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:55:58.982486 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:58.982450 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56"] Jul 08 19:55:58.982931 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:58.982916 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b6c0bba-dd1a-4595-9c28-f99ca1c9c935" containerName="istio-proxy" Jul 08 19:55:58.986065 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:58.986048 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:58.989496 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:58.989468 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c4f984b0\"/\"e2e-lifecycle-model-c4f984b0-kserve-self-signed-certs\"" Jul 08 19:55:58.989632 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:58.989529 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c4f984b0\"/\"openshift-service-ca.crt\"" Jul 08 19:55:58.989632 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:58.989547 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-c4f984b0\"/\"kube-root-ca.crt\"" Jul 08 19:55:58.989632 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:58.989562 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-c4f984b0\"/\"default-dockercfg-2tz42\"" Jul 08 19:55:58.996940 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:58.996913 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56"] Jul 08 19:55:59.007696 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.007663 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-dshm\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.007897 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.007703 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-home\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.007897 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.007738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tmp-dir\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.007897 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.007795 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tls-certs\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.007897 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.007831 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-model-cache\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.007897 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.007873 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzcrw\" (UniqueName: \"kubernetes.io/projected/0040bdb6-d5d0-4b83-b071-c35b40932e2c-kube-api-access-dzcrw\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.108889 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.108841 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-dshm\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.108889 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.108883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-home\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.109173 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.108921 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tmp-dir\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.109173 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.108962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tls-certs\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.109173 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.108985 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-model-cache\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.109332 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.109221 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzcrw\" (UniqueName: \"kubernetes.io/projected/0040bdb6-d5d0-4b83-b071-c35b40932e2c-kube-api-access-dzcrw\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.109427 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.109399 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-home\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.109486 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.109430 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tmp-dir\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.109486 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.109446 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-model-cache\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.111278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.111257 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-dshm\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.111453 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.111433 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tls-certs\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.116935 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.116908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzcrw\" (UniqueName: \"kubernetes.io/projected/0040bdb6-d5d0-4b83-b071-c35b40932e2c-kube-api-access-dzcrw\") pod \"e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.296237 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.296191 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:55:59.437388 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:55:59.437359 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56"] Jul 08 19:55:59.439835 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:55:59.439807 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0040bdb6_d5d0_4b83_b071_c35b40932e2c.slice/crio-c391daf62ee672c9adfa6c667ee9b729d5221ad6f27f61f141b769dacf008b91 WatchSource:0}: Error finding container c391daf62ee672c9adfa6c667ee9b729d5221ad6f27f61f141b769dacf008b91: Status 404 returned error can't find the container with id c391daf62ee672c9adfa6c667ee9b729d5221ad6f27f61f141b769dacf008b91 Jul 08 19:56:00.201355 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:00.201317 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" event={"ID":"0040bdb6-d5d0-4b83-b071-c35b40932e2c","Type":"ContainerStarted","Data":"bd0cdd78c19c590ce238dc4b6bbe7188227bb7e3a688d5d6822b5b95f1c47abb"} Jul 08 19:56:00.201355 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:00.201363 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" event={"ID":"0040bdb6-d5d0-4b83-b071-c35b40932e2c","Type":"ContainerStarted","Data":"c391daf62ee672c9adfa6c667ee9b729d5221ad6f27f61f141b769dacf008b91"} Jul 08 19:56:00.222503 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:00.222447 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" podStartSLOduration=2.222429315 podStartE2EDuration="2.222429315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:55:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:56:00.221093422 +0000 UTC m=+1921.878532029" watchObservedRunningTime="2026-07-08 19:56:00.222429315 +0000 UTC m=+1921.879867921" Jul 08 19:56:09.297303 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:09.297262 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:56:09.297808 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:09.297319 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:56:09.317083 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:09.317046 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:56:10.270318 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:10.270284 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:56:29.198391 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:29.198356 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56"] Jul 08 19:56:44.218999 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.218934 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" podUID="0040bdb6-d5d0-4b83-b071-c35b40932e2c" containerName="main" containerID="cri-o://bd0cdd78c19c590ce238dc4b6bbe7188227bb7e3a688d5d6822b5b95f1c47abb" gracePeriod=45 Jul 08 19:56:44.457797 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.457764 2573 generic.go:356] "Generic (PLEG): container finished" podID="0040bdb6-d5d0-4b83-b071-c35b40932e2c" containerID="bd0cdd78c19c590ce238dc4b6bbe7188227bb7e3a688d5d6822b5b95f1c47abb" exitCode=0 Jul 08 19:56:44.457797 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.457787 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" event={"ID":"0040bdb6-d5d0-4b83-b071-c35b40932e2c","Type":"ContainerDied","Data":"bd0cdd78c19c590ce238dc4b6bbe7188227bb7e3a688d5d6822b5b95f1c47abb"} Jul 08 19:56:44.472528 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.472470 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:56:44.542679 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.542647 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-model-cache\") pod \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " Jul 08 19:56:44.542891 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.542737 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-home\") pod \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " Jul 08 19:56:44.542891 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.542765 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-dshm\") pod \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " Jul 08 19:56:44.542891 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.542791 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tmp-dir\") pod \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " Jul 08 19:56:44.542891 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.542808 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tls-certs\") pod \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " Jul 08 19:56:44.542891 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.542834 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzcrw\" (UniqueName: \"kubernetes.io/projected/0040bdb6-d5d0-4b83-b071-c35b40932e2c-kube-api-access-dzcrw\") pod \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\" (UID: \"0040bdb6-d5d0-4b83-b071-c35b40932e2c\") " Jul 08 19:56:44.543186 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.543010 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-model-cache" (OuterVolumeSpecName: "model-cache") pod "0040bdb6-d5d0-4b83-b071-c35b40932e2c" (UID: "0040bdb6-d5d0-4b83-b071-c35b40932e2c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:56:44.543186 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.543068 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-home" (OuterVolumeSpecName: "home") pod "0040bdb6-d5d0-4b83-b071-c35b40932e2c" (UID: "0040bdb6-d5d0-4b83-b071-c35b40932e2c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:56:44.543186 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.543110 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0040bdb6-d5d0-4b83-b071-c35b40932e2c" (UID: "0040bdb6-d5d0-4b83-b071-c35b40932e2c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:56:44.543337 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.543207 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-home\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:56:44.543337 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.543225 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tmp-dir\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:56:44.543337 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.543241 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-model-cache\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:56:44.545154 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.545121 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0040bdb6-d5d0-4b83-b071-c35b40932e2c-kube-api-access-dzcrw" (OuterVolumeSpecName: "kube-api-access-dzcrw") pod "0040bdb6-d5d0-4b83-b071-c35b40932e2c" (UID: "0040bdb6-d5d0-4b83-b071-c35b40932e2c"). InnerVolumeSpecName "kube-api-access-dzcrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:56:44.545258 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.545163 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-dshm" (OuterVolumeSpecName: "dshm") pod "0040bdb6-d5d0-4b83-b071-c35b40932e2c" (UID: "0040bdb6-d5d0-4b83-b071-c35b40932e2c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:56:44.545258 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.545216 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0040bdb6-d5d0-4b83-b071-c35b40932e2c" (UID: "0040bdb6-d5d0-4b83-b071-c35b40932e2c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:56:44.643742 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.643700 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzcrw\" (UniqueName: \"kubernetes.io/projected/0040bdb6-d5d0-4b83-b071-c35b40932e2c-kube-api-access-dzcrw\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:56:44.643742 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.643738 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0040bdb6-d5d0-4b83-b071-c35b40932e2c-dshm\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:56:44.643742 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:44.643751 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0040bdb6-d5d0-4b83-b071-c35b40932e2c-tls-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:56:45.465615 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:45.465579 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" event={"ID":"0040bdb6-d5d0-4b83-b071-c35b40932e2c","Type":"ContainerDied","Data":"c391daf62ee672c9adfa6c667ee9b729d5221ad6f27f61f141b769dacf008b91"} Jul 08 19:56:45.465615 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:45.465609 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56" Jul 08 19:56:45.466165 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:45.465634 2573 scope.go:117] "RemoveContainer" containerID="bd0cdd78c19c590ce238dc4b6bbe7188227bb7e3a688d5d6822b5b95f1c47abb" Jul 08 19:56:45.483304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:45.483270 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56"] Jul 08 19:56:45.488118 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:45.488090 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-c4f984b0/e2e-lifecycle-model-c4f984b0-kserve-5d8f99db99-zgk56"] Jul 08 19:56:46.567364 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:56:46.567333 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0040bdb6-d5d0-4b83-b071-c35b40932e2c" path="/var/lib/kubelet/pods/0040bdb6-d5d0-4b83-b071-c35b40932e2c/volumes" Jul 08 19:57:36.921175 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:36.921137 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm"] Jul 08 19:57:36.922211 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:36.921401 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" containerID="cri-o://7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42" gracePeriod=30 Jul 08 19:57:42.188442 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.188373 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:57:42.285524 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285485 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-podinfo\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.285705 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285547 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-data\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.285705 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285591 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-token\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.285705 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285614 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-socket\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.285885 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285735 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-certs\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.285885 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285760 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istiod-ca-cert\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.285885 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285796 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-envoy\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.285885 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285858 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-credential-socket\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.285885 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285886 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx48t\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-kube-api-access-nx48t\") pod \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\" (UID: \"620b6f0a-aab7-4cad-9012-8b1cce13dc6f\") " Jul 08 19:57:42.286199 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.285976 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:57:42.286199 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.286183 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.286505 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.286477 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-data" (OuterVolumeSpecName: "istio-data") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:57:42.286664 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.286637 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:57:42.286761 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.286684 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:57:42.286823 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.286617 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:57:42.287985 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.287954 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 19:57:42.288332 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.288305 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-token" (OuterVolumeSpecName: "istio-token") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:57:42.288558 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.288529 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-kube-api-access-nx48t" (OuterVolumeSpecName: "kube-api-access-nx48t") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "kube-api-access-nx48t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:57:42.288944 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.288921 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "620b6f0a-aab7-4cad-9012-8b1cce13dc6f" (UID: "620b6f0a-aab7-4cad-9012-8b1cce13dc6f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:57:42.387238 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.387198 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.387238 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.387228 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.387238 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.387237 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.387238 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.387246 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.387509 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.387256 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx48t\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-kube-api-access-nx48t\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.387509 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.387265 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.387509 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.387272 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.387509 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.387279 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/620b6f0a-aab7-4cad-9012-8b1cce13dc6f-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 19:57:42.782611 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.782569 2573 generic.go:356] "Generic (PLEG): container finished" podID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerID="7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42" exitCode=0 Jul 08 19:57:42.782816 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.782629 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" Jul 08 19:57:42.782816 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.782628 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" event={"ID":"620b6f0a-aab7-4cad-9012-8b1cce13dc6f","Type":"ContainerDied","Data":"7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42"} Jul 08 19:57:42.782816 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.782764 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm" event={"ID":"620b6f0a-aab7-4cad-9012-8b1cce13dc6f","Type":"ContainerDied","Data":"b9e915092ae1260b5b9f3ffd0164205e0300ede46c267f8ab53efe5ed4608d36"} Jul 08 19:57:42.782816 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.782796 2573 scope.go:117] "RemoveContainer" containerID="7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42" Jul 08 19:57:42.794398 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.794374 2573 scope.go:117] "RemoveContainer" containerID="ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0" Jul 08 19:57:42.800064 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.800038 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm"] Jul 08 19:57:42.803663 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.803644 2573 scope.go:117] "RemoveContainer" containerID="7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42" Jul 08 19:57:42.804071 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:57:42.804044 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42\": container with ID starting with 7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42 not found: ID does not exist" containerID="7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42" Jul 08 19:57:42.804166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.804113 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42"} err="failed to get container status \"7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42\": rpc error: code = NotFound desc = could not find container \"7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42\": container with ID starting with 7f850e1b354c6a44a279aafdaf3be455c13fc568fa6417ebab07a40286f8ee42 not found: ID does not exist" Jul 08 19:57:42.804166 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.804141 2573 scope.go:117] "RemoveContainer" containerID="ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0" Jul 08 19:57:42.804563 ip-10-0-143-40 kubenswrapper[2573]: E0708 19:57:42.804536 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0\": container with ID starting with ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0 not found: ID does not exist" containerID="ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0" Jul 08 19:57:42.804701 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.804571 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0"} err="failed to get container status \"ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0\": rpc error: code = NotFound desc = could not find container \"ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0\": container with ID starting with ffcbd92d3b5b7d3dbca66bc0718b2a303b434fb9330e522bfe8966c1cb3f3da0 not found: ID does not exist" Jul 08 19:57:42.814280 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:42.814256 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c4f984b0-openshift-default-7976f5cfc5-nhjkm"] Jul 08 19:57:44.567763 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:57:44.567730 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" path="/var/lib/kubelet/pods/620b6f0a-aab7-4cad-9012-8b1cce13dc6f/volumes" Jul 08 19:58:32.847831 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.847751 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9"] Jul 08 19:58:32.848304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.848119 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" Jul 08 19:58:32.848304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.848131 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" Jul 08 19:58:32.848304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.848142 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0040bdb6-d5d0-4b83-b071-c35b40932e2c" containerName="main" Jul 08 19:58:32.848304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.848148 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0040bdb6-d5d0-4b83-b071-c35b40932e2c" containerName="main" Jul 08 19:58:32.848304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.848156 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" Jul 08 19:58:32.848304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.848162 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" Jul 08 19:58:32.848304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.848225 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" Jul 08 19:58:32.848304 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.848232 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0040bdb6-d5d0-4b83-b071-c35b40932e2c" containerName="main" Jul 08 19:58:32.851162 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.851145 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.854083 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.854063 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d7248b-openshift-default-dockercfg-pscgj\"" Jul 08 19:58:32.874201 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.874168 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9"] Jul 08 19:58:32.923646 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923602 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-data\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.923842 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923649 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-podinfo\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.923842 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923693 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-socket\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.923842 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923720 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-token\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.923842 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923795 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-envoy\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.923842 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923816 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-credential-socket\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.924139 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923920 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w84m4\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-kube-api-access-w84m4\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.924139 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923947 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-certs\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:32.924139 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:32.923962 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istiod-ca-cert\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.024787 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.024744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-data\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.024787 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.024796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-podinfo\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.024820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-socket\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.024842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-token\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.024878 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-envoy\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.024903 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-credential-socket\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.024976 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w84m4\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-kube-api-access-w84m4\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.025000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-certs\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025084 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.025059 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istiod-ca-cert\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025475 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.025244 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-data\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025475 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.025371 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-credential-socket\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025475 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.025425 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-certs\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025662 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.025637 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-socket\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.025807 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.025789 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istiod-ca-cert\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.027550 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.027524 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-envoy\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.027679 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.027655 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-podinfo\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.038909 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.038878 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w84m4\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-kube-api-access-w84m4\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.039389 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.039373 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-token\") pod \"e2e-inf-a-d7248b-openshift-default-869865687f-vggd9\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.161604 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.161514 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:33.312278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.312246 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9"] Jul 08 19:58:33.313780 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:58:33.313748 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639729bd_59e7_4e4e_8ae0_7ad8544006b8.slice/crio-0288ff16f6c4bf4aca83ed1c1c0ab9e67734c098d7646a157ded8a8014b4cb76 WatchSource:0}: Error finding container 0288ff16f6c4bf4aca83ed1c1c0ab9e67734c098d7646a157ded8a8014b4cb76: Status 404 returned error can't find the container with id 0288ff16f6c4bf4aca83ed1c1c0ab9e67734c098d7646a157ded8a8014b4cb76 Jul 08 19:58:33.315804 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.315773 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:33.315885 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.315837 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:33.315885 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:33.315870 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:34.054342 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:34.054308 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" event={"ID":"639729bd-59e7-4e4e-8ae0-7ad8544006b8","Type":"ContainerStarted","Data":"2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72"} Jul 08 19:58:34.054742 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:34.054363 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" event={"ID":"639729bd-59e7-4e4e-8ae0-7ad8544006b8","Type":"ContainerStarted","Data":"0288ff16f6c4bf4aca83ed1c1c0ab9e67734c098d7646a157ded8a8014b4cb76"} Jul 08 19:58:34.077608 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:34.077555 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" podStartSLOduration=2.077538991 podStartE2EDuration="2.077538991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:58:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:58:34.07420789 +0000 UTC m=+2075.731646513" watchObservedRunningTime="2026-07-08 19:58:34.077538991 +0000 UTC m=+2075.734977597" Jul 08 19:58:34.162185 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:34.162146 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:34.167175 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:34.167147 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:35.060066 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:35.060015 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:35.061154 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:35.061133 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:41.183127 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.183085 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz"] Jul 08 19:58:41.183785 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.183761 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="620b6f0a-aab7-4cad-9012-8b1cce13dc6f" containerName="istio-proxy" Jul 08 19:58:41.187484 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.187462 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.189963 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.189942 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-deba89-openshift-default-dockercfg-fsvp5\"" Jul 08 19:58:41.200096 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.200070 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz"] Jul 08 19:58:41.309128 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309086 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-socket\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.309321 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309135 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-data\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.309321 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309168 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-podinfo\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.309321 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309206 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkphl\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-kube-api-access-pkphl\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.309445 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309326 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-certs\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.309445 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309359 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-envoy\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.309445 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309418 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee0a3707-4a63-466b-920e-512b1f13e9ce-istiod-ca-cert\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.309445 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309436 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-credential-socket\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.309593 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.309456 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-token\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.410890 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.410851 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-socket\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.410892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-data\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.410930 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-podinfo\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.410953 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkphl\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-kube-api-access-pkphl\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-certs\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411133 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411087 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-envoy\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411260 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee0a3707-4a63-466b-920e-512b1f13e9ce-istiod-ca-cert\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411309 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-credential-socket\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411332 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-socket\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-token\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411425 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411384 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-data\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.411695 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411436 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-certs\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.412008 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.411984 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-credential-socket\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.412134 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.412004 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee0a3707-4a63-466b-920e-512b1f13e9ce-istiod-ca-cert\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.413513 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.413491 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-podinfo\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.413616 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.413599 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-envoy\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.419338 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.419312 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-token\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.419422 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.419345 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkphl\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-kube-api-access-pkphl\") pod \"e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.498534 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.498509 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:41.648526 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.648266 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz"] Jul 08 19:58:41.651035 ip-10-0-143-40 kubenswrapper[2573]: W0708 19:58:41.650996 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0a3707_4a63_466b_920e_512b1f13e9ce.slice/crio-99fa39b0eec58d6727d459ea82bbb6bd010d437ab55cecdb274685eea880c78d WatchSource:0}: Error finding container 99fa39b0eec58d6727d459ea82bbb6bd010d437ab55cecdb274685eea880c78d: Status 404 returned error can't find the container with id 99fa39b0eec58d6727d459ea82bbb6bd010d437ab55cecdb274685eea880c78d Jul 08 19:58:41.653612 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.653581 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:41.653720 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.653653 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:41.653720 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:41.653682 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:42.099843 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:42.099810 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" event={"ID":"ee0a3707-4a63-466b-920e-512b1f13e9ce","Type":"ContainerStarted","Data":"ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5"} Jul 08 19:58:42.100044 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:42.099859 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" event={"ID":"ee0a3707-4a63-466b-920e-512b1f13e9ce","Type":"ContainerStarted","Data":"99fa39b0eec58d6727d459ea82bbb6bd010d437ab55cecdb274685eea880c78d"} Jul 08 19:58:42.126836 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:42.126774 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" podStartSLOduration=1.126752095 podStartE2EDuration="1.126752095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:58:42.125687041 +0000 UTC m=+2083.783125646" watchObservedRunningTime="2026-07-08 19:58:42.126752095 +0000 UTC m=+2083.784190702" Jul 08 19:58:42.499696 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:42.499662 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:42.505072 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:42.505047 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:43.104940 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:43.104905 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:43.105944 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:43.105924 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:58:51.102233 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:51.102186 2573 patch_prober.go:28] interesting pod/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 19:58:51.102664 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:51.102270 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 19:58:52.175698 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:52.175663 2573 generic.go:356] "Generic (PLEG): container finished" podID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerID="2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72" exitCode=0 Jul 08 19:58:52.176147 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:52.175740 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" event={"ID":"639729bd-59e7-4e4e-8ae0-7ad8544006b8","Type":"ContainerDied","Data":"2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72"} Jul 08 19:58:52.176223 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:52.176208 2573 scope.go:117] "RemoveContainer" containerID="2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72" Jul 08 19:58:52.177356 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:52.177330 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:52.177419 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:52.177399 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:52.177465 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:52.177433 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:58:53.185638 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:53.185605 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" event={"ID":"639729bd-59e7-4e4e-8ae0-7ad8544006b8","Type":"ContainerStarted","Data":"675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf"} Jul 08 19:58:54.162759 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:54.162714 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:54.162994 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:54.162859 2573 patch_prober.go:28] interesting pod/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 08 19:58:54.162994 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:54.162905 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 08 19:58:55.162449 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:55.162409 2573 patch_prober.go:28] interesting pod/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 08 19:58:55.163521 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:55.163073 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 08 19:58:56.163004 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:56.162956 2573 patch_prober.go:28] interesting pod/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 08 19:58:56.163538 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:56.163055 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 08 19:58:57.165871 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:57.165834 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:57.166345 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:57.166279 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:57.167099 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:57.167080 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 19:58:59.146268 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:59.146221 2573 patch_prober.go:28] interesting pod/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 08 19:58:59.146636 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:58:59.146306 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 08 19:59:00.224366 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:00.224270 2573 generic.go:356] "Generic (PLEG): container finished" podID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerID="ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5" exitCode=0 Jul 08 19:59:00.224366 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:00.224344 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" event={"ID":"ee0a3707-4a63-466b-920e-512b1f13e9ce","Type":"ContainerDied","Data":"ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5"} Jul 08 19:59:00.224818 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:00.224745 2573 scope.go:117] "RemoveContainer" containerID="ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5" Jul 08 19:59:00.225794 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:00.225763 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:59:00.226135 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:00.225837 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:59:00.226135 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:00.225871 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 19:59:01.235665 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:01.235619 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" event={"ID":"ee0a3707-4a63-466b-920e-512b1f13e9ce","Type":"ContainerStarted","Data":"21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb"} Jul 08 19:59:01.499673 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:01.499633 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:59:01.499956 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:01.499933 2573 patch_prober.go:28] interesting pod/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 08 19:59:01.500017 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:01.499987 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 08 19:59:02.498981 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:02.498942 2573 patch_prober.go:28] interesting pod/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 08 19:59:02.499417 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:02.498998 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 08 19:59:03.499044 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:03.498997 2573 patch_prober.go:28] interesting pod/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 08 19:59:03.499471 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:03.499119 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 08 19:59:04.499138 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:04.499084 2573 patch_prober.go:28] interesting pod/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 08 19:59:04.499551 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:04.499159 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 08 19:59:05.502859 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:05.502828 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:59:05.503278 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:05.503266 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 19:59:05.504117 ip-10-0-143-40 kubenswrapper[2573]: I0708 19:59:05.504097 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 20:00:56.977665 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:00:56.977632 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9"] Jul 08 20:00:56.978228 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:00:56.977865 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" containerID="cri-o://675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf" gracePeriod=30 Jul 08 20:00:57.168403 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:00:57.168359 2573 patch_prober.go:28] interesting pod/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 20:00:57.168570 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:00:57.168431 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 20:01:02.237289 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.237263 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 20:01:02.324942 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.324907 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-token\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325151 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.324980 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-data\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325151 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325000 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w84m4\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-kube-api-access-w84m4\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325151 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325066 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-socket\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325151 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325100 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-credential-socket\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325331 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325262 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-envoy\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325331 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325311 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-podinfo\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325444 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325352 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-certs\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325444 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325379 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:02.325444 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325394 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:02.325444 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325405 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istiod-ca-cert\") pod \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\" (UID: \"639729bd-59e7-4e4e-8ae0-7ad8544006b8\") " Jul 08 20:01:02.325938 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325706 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:01:02.325938 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325820 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.325938 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325844 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.325938 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325857 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.325938 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325877 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:02.325938 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.325893 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:02.327557 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.327512 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:02.327657 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.327592 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:01:02.327719 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.327676 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:01:02.327719 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.327690 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-kube-api-access-w84m4" (OuterVolumeSpecName: "kube-api-access-w84m4") pod "639729bd-59e7-4e4e-8ae0-7ad8544006b8" (UID: "639729bd-59e7-4e4e-8ae0-7ad8544006b8"). InnerVolumeSpecName "kube-api-access-w84m4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:01:02.426512 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.426469 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.426512 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.426509 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.426741 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.426523 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.426741 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.426536 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.426741 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.426547 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/639729bd-59e7-4e4e-8ae0-7ad8544006b8-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.426741 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.426560 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w84m4\" (UniqueName: \"kubernetes.io/projected/639729bd-59e7-4e4e-8ae0-7ad8544006b8-kube-api-access-w84m4\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:02.926789 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.926681 2573 generic.go:356] "Generic (PLEG): container finished" podID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerID="675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf" exitCode=0 Jul 08 20:01:02.926789 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.926753 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" Jul 08 20:01:02.927050 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.926743 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" event={"ID":"639729bd-59e7-4e4e-8ae0-7ad8544006b8","Type":"ContainerDied","Data":"675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf"} Jul 08 20:01:02.927050 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.926873 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9" event={"ID":"639729bd-59e7-4e4e-8ae0-7ad8544006b8","Type":"ContainerDied","Data":"0288ff16f6c4bf4aca83ed1c1c0ab9e67734c098d7646a157ded8a8014b4cb76"} Jul 08 20:01:02.927050 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.926906 2573 scope.go:117] "RemoveContainer" containerID="675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf" Jul 08 20:01:02.936439 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.936420 2573 scope.go:117] "RemoveContainer" containerID="2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72" Jul 08 20:01:02.945169 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.945146 2573 scope.go:117] "RemoveContainer" containerID="675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf" Jul 08 20:01:02.945509 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:01:02.945482 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf\": container with ID starting with 675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf not found: ID does not exist" containerID="675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf" Jul 08 20:01:02.945603 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.945516 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf"} err="failed to get container status \"675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf\": rpc error: code = NotFound desc = could not find container \"675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf\": container with ID starting with 675f6660ae95963c77b0164fea06a7e0d549143241f38854203b721113d744cf not found: ID does not exist" Jul 08 20:01:02.945603 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.945536 2573 scope.go:117] "RemoveContainer" containerID="2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72" Jul 08 20:01:02.945787 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:01:02.945772 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72\": container with ID starting with 2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72 not found: ID does not exist" containerID="2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72" Jul 08 20:01:02.945827 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.945793 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72"} err="failed to get container status \"2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72\": rpc error: code = NotFound desc = could not find container \"2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72\": container with ID starting with 2a6bbbad1d93bce0a3e8f0322106e959f8f42dc8cd594476d84db1f823606e72 not found: ID does not exist" Jul 08 20:01:02.946084 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.946067 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9"] Jul 08 20:01:02.949426 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:02.949404 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d7248b-openshift-default-869865687f-vggd9"] Jul 08 20:01:04.330922 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:04.330883 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz"] Jul 08 20:01:04.331835 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:04.331771 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" containerID="cri-o://21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb" gracePeriod=30 Jul 08 20:01:04.567473 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:04.567437 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" path="/var/lib/kubelet/pods/639729bd-59e7-4e4e-8ae0-7ad8544006b8/volumes" Jul 08 20:01:05.242017 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.241984 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9"] Jul 08 20:01:05.242449 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.242429 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" Jul 08 20:01:05.242533 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.242454 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" Jul 08 20:01:05.242533 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.242473 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" Jul 08 20:01:05.242533 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.242481 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" Jul 08 20:01:05.242688 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.242617 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" Jul 08 20:01:05.242688 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.242632 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="639729bd-59e7-4e4e-8ae0-7ad8544006b8" containerName="istio-proxy" Jul 08 20:01:05.244867 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.244851 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.247269 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.247236 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6abdf4-openshift-default-dockercfg-69dvc\"" Jul 08 20:01:05.257389 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.257365 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9"] Jul 08 20:01:05.355325 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355287 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8977f8cd-05d6-4127-8f03-90271b9c588f-istiod-ca-cert\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.355795 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355341 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-socket\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.355795 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355364 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-data\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.355795 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355389 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-envoy\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.355795 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355440 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-podinfo\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.355795 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355532 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-certs\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.355795 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355567 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-credential-socket\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.355795 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355598 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzcck\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-kube-api-access-zzcck\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.355795 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.355675 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-token\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456648 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456607 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8977f8cd-05d6-4127-8f03-90271b9c588f-istiod-ca-cert\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456660 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-socket\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-data\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-envoy\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-podinfo\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456749 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-certs\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-credential-socket\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456796 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzcck\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-kube-api-access-zzcck\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.456896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.456823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-token\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.457378 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.457115 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-socket\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.457378 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.457347 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-certs\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.457633 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.457608 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8977f8cd-05d6-4127-8f03-90271b9c588f-istiod-ca-cert\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.457741 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.457649 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-data\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.457741 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.457714 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-credential-socket\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.459270 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.459248 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-envoy\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.459481 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.459462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-podinfo\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.465126 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.465102 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzcck\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-kube-api-access-zzcck\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.465211 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.465136 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-token\") pod \"e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.505813 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.505730 2573 patch_prober.go:28] interesting pod/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 20:01:05.505949 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.505819 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 20:01:05.555621 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.555586 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:05.694610 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.694582 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9"] Jul 08 20:01:05.696783 ip-10-0-143-40 kubenswrapper[2573]: W0708 20:01:05.696752 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8977f8cd_05d6_4127_8f03_90271b9c588f.slice/crio-ba3b4e7ecf8d68ec31d4d9551e830529a908ba8682a27a24f8cc062702c1530f WatchSource:0}: Error finding container ba3b4e7ecf8d68ec31d4d9551e830529a908ba8682a27a24f8cc062702c1530f: Status 404 returned error can't find the container with id ba3b4e7ecf8d68ec31d4d9551e830529a908ba8682a27a24f8cc062702c1530f Jul 08 20:01:05.699091 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.699056 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:05.699171 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.699146 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:05.699210 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.699185 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:05.948352 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.948245 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" event={"ID":"8977f8cd-05d6-4127-8f03-90271b9c588f","Type":"ContainerStarted","Data":"312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236"} Jul 08 20:01:05.948352 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.948324 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" event={"ID":"8977f8cd-05d6-4127-8f03-90271b9c588f","Type":"ContainerStarted","Data":"ba3b4e7ecf8d68ec31d4d9551e830529a908ba8682a27a24f8cc062702c1530f"} Jul 08 20:01:05.971957 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:05.971902 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" podStartSLOduration=0.971884468 podStartE2EDuration="971.884468ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:01:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:01:05.96954427 +0000 UTC m=+2227.626982872" watchObservedRunningTime="2026-07-08 20:01:05.971884468 +0000 UTC m=+2227.629323073" Jul 08 20:01:06.556480 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:06.556439 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:06.558132 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:06.558104 2573 patch_prober.go:28] interesting pod/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 08 20:01:06.558255 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:06.558156 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 08 20:01:07.560114 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:07.560083 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:07.960526 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:07.960440 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:07.961449 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:07.961424 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:09.593609 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.593534 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 20:01:09.694681 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.694649 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee0a3707-4a63-466b-920e-512b1f13e9ce-istiod-ca-cert\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.694681 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.694686 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-certs\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.694941 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.694726 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkphl\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-kube-api-access-pkphl\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.694941 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.694755 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-socket\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.694941 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.694797 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-data\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.694941 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.694811 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-token\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.694941 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.694913 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-podinfo\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.695246 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.694963 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-envoy\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.695246 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.695065 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-credential-socket\") pod \"ee0a3707-4a63-466b-920e-512b1f13e9ce\" (UID: \"ee0a3707-4a63-466b-920e-512b1f13e9ce\") " Jul 08 20:01:09.695430 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.695063 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:09.695430 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.695382 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:09.695430 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.695112 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee0a3707-4a63-466b-920e-512b1f13e9ce-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:01:09.695430 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.695203 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:09.695860 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.695832 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:09.697147 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.697121 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:01:09.697235 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.697173 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:01:09.697235 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.697186 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-kube-api-access-pkphl" (OuterVolumeSpecName: "kube-api-access-pkphl") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "kube-api-access-pkphl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:01:09.697235 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.697218 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee0a3707-4a63-466b-920e-512b1f13e9ce" (UID: "ee0a3707-4a63-466b-920e-512b1f13e9ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:09.796473 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796433 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.796473 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796464 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.796473 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796473 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.796473 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796482 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.796745 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796491 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.796745 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796501 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee0a3707-4a63-466b-920e-512b1f13e9ce-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.796745 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796510 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.796745 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796517 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkphl\" (UniqueName: \"kubernetes.io/projected/ee0a3707-4a63-466b-920e-512b1f13e9ce-kube-api-access-pkphl\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.796745 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.796526 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee0a3707-4a63-466b-920e-512b1f13e9ce-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:09.973243 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.973149 2573 generic.go:356] "Generic (PLEG): container finished" podID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerID="21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb" exitCode=0 Jul 08 20:01:09.973243 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.973203 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" Jul 08 20:01:09.973464 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.973242 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" event={"ID":"ee0a3707-4a63-466b-920e-512b1f13e9ce","Type":"ContainerDied","Data":"21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb"} Jul 08 20:01:09.973464 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.973303 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz" event={"ID":"ee0a3707-4a63-466b-920e-512b1f13e9ce","Type":"ContainerDied","Data":"99fa39b0eec58d6727d459ea82bbb6bd010d437ab55cecdb274685eea880c78d"} Jul 08 20:01:09.973464 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.973333 2573 scope.go:117] "RemoveContainer" containerID="21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb" Jul 08 20:01:09.985876 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.985844 2573 scope.go:117] "RemoveContainer" containerID="ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5" Jul 08 20:01:09.994706 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.994677 2573 scope.go:117] "RemoveContainer" containerID="21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb" Jul 08 20:01:09.994993 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:01:09.994976 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb\": container with ID starting with 21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb not found: ID does not exist" containerID="21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb" Jul 08 20:01:09.995061 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.995002 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb"} err="failed to get container status \"21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb\": rpc error: code = NotFound desc = could not find container \"21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb\": container with ID starting with 21c5be3f8756dd76d2287e0b880a66a45db8d1c84d906ba3881a1f8e28f4f0bb not found: ID does not exist" Jul 08 20:01:09.995061 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.995034 2573 scope.go:117] "RemoveContainer" containerID="ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5" Jul 08 20:01:09.995286 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:01:09.995267 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5\": container with ID starting with ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5 not found: ID does not exist" containerID="ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5" Jul 08 20:01:09.995338 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:09.995290 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5"} err="failed to get container status \"ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5\": rpc error: code = NotFound desc = could not find container \"ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5\": container with ID starting with ceba1ece59ae66311be1e298f4a4d7af37ee6a71349b25aeae0d189dd7ccd0b5 not found: ID does not exist" Jul 08 20:01:10.001399 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:10.001375 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz"] Jul 08 20:01:10.005177 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:10.005154 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-deba89-openshift-default-74448c4ffc-xkzgz"] Jul 08 20:01:10.567699 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:10.567663 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" path="/var/lib/kubelet/pods/ee0a3707-4a63-466b-920e-512b1f13e9ce/volumes" Jul 08 20:01:13.164616 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.164578 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh"] Jul 08 20:01:13.165193 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.165174 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" Jul 08 20:01:13.165245 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.165198 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" Jul 08 20:01:13.165245 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.165223 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" Jul 08 20:01:13.165245 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.165232 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" Jul 08 20:01:13.165361 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.165346 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" Jul 08 20:01:13.165420 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.165366 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee0a3707-4a63-466b-920e-512b1f13e9ce" containerName="istio-proxy" Jul 08 20:01:13.168204 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.168187 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.170738 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.170711 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-b7fdb0-openshift-default-dockercfg-pgxrf\"" Jul 08 20:01:13.180407 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.180385 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh"] Jul 08 20:01:13.330364 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330320 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj2nb\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-kube-api-access-cj2nb\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.330560 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330378 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-istio-token\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.330560 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330432 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-envoy\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.330560 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330500 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79991458-b7db-46f4-8d3a-703762362408-istiod-ca-cert\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.330560 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330555 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79991458-b7db-46f4-8d3a-703762362408-istio-podinfo\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.330807 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330613 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-certs\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.330807 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330656 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-data\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.330807 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330723 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-socket\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.330807 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.330748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-credential-socket\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.431563 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431457 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-istio-token\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.431563 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431511 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-envoy\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.431563 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431539 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79991458-b7db-46f4-8d3a-703762362408-istiod-ca-cert\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.431859 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79991458-b7db-46f4-8d3a-703762362408-istio-podinfo\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.431859 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431736 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-certs\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.431859 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-data\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.431859 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431840 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-socket\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.431859 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431866 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-credential-socket\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.432197 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.431937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj2nb\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-kube-api-access-cj2nb\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.432303 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.432278 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-certs\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.432471 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.432336 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79991458-b7db-46f4-8d3a-703762362408-istiod-ca-cert\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.432558 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.432510 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-data\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.433165 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.433006 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-credential-socket\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.433165 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.433122 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-socket\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.434885 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.434854 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-envoy\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.435150 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.435126 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79991458-b7db-46f4-8d3a-703762362408-istio-podinfo\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.439549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.439521 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-istio-token\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.439644 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.439586 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj2nb\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-kube-api-access-cj2nb\") pod \"e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.480004 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.479962 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:13.629460 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.629431 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh"] Jul 08 20:01:13.632830 ip-10-0-143-40 kubenswrapper[2573]: W0708 20:01:13.632800 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79991458_b7db_46f4_8d3a_703762362408.slice/crio-c6d5af672f48e758f47b287bf07ce05bfa3cfc37b148d73afc008316de4f668f WatchSource:0}: Error finding container c6d5af672f48e758f47b287bf07ce05bfa3cfc37b148d73afc008316de4f668f: Status 404 returned error can't find the container with id c6d5af672f48e758f47b287bf07ce05bfa3cfc37b148d73afc008316de4f668f Jul 08 20:01:13.635383 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.635346 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:13.635493 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.635423 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:13.635493 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.635466 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:13.997470 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.997431 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" event={"ID":"79991458-b7db-46f4-8d3a-703762362408","Type":"ContainerStarted","Data":"ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a"} Jul 08 20:01:13.997673 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:13.997496 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" event={"ID":"79991458-b7db-46f4-8d3a-703762362408","Type":"ContainerStarted","Data":"c6d5af672f48e758f47b287bf07ce05bfa3cfc37b148d73afc008316de4f668f"} Jul 08 20:01:14.019623 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:14.019475 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" podStartSLOduration=1.019453069 podStartE2EDuration="1.019453069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:01:14.01922164 +0000 UTC m=+2235.676660247" watchObservedRunningTime="2026-07-08 20:01:14.019453069 +0000 UTC m=+2235.676891678" Jul 08 20:01:14.480566 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:14.480465 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:14.485302 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:14.485273 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:15.004161 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:15.004130 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:15.005211 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:15.005187 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:22.051881 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:22.051690 2573 generic.go:356] "Generic (PLEG): container finished" podID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerID="312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236" exitCode=0 Jul 08 20:01:22.051881 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:22.051782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" event={"ID":"8977f8cd-05d6-4127-8f03-90271b9c588f","Type":"ContainerDied","Data":"312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236"} Jul 08 20:01:22.052796 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:22.052759 2573 scope.go:117] "RemoveContainer" containerID="312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236" Jul 08 20:01:22.054933 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:22.054892 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:22.055064 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:22.055018 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:22.055140 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:22.055099 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:22.961133 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:22.961089 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:23.060077 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:23.060014 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" event={"ID":"8977f8cd-05d6-4127-8f03-90271b9c588f","Type":"ContainerStarted","Data":"f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f"} Jul 08 20:01:23.556767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:23.556725 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:23.557901 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:23.557876 2573 patch_prober.go:28] interesting pod/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 08 20:01:23.558048 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:23.557930 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 08 20:01:24.556300 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:24.556252 2573 patch_prober.go:28] interesting pod/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 08 20:01:24.556843 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:24.556324 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 08 20:01:25.557051 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:25.556988 2573 patch_prober.go:28] interesting pod/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 08 20:01:25.557654 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:25.557619 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 08 20:01:26.556849 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:26.556811 2573 patch_prober.go:28] interesting pod/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 08 20:01:26.557050 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:26.556876 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 08 20:01:27.556331 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:27.556288 2573 patch_prober.go:28] interesting pod/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 08 20:01:27.556784 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:27.556361 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 08 20:01:28.570356 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:28.570319 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:28.570962 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:28.570748 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:28.571896 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:28.571869 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:30.005439 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:30.005404 2573 patch_prober.go:28] interesting pod/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 08 20:01:30.005927 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:30.005465 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 08 20:01:30.114698 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:30.114659 2573 generic.go:356] "Generic (PLEG): container finished" podID="79991458-b7db-46f4-8d3a-703762362408" containerID="ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a" exitCode=0 Jul 08 20:01:30.114901 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:30.114733 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" event={"ID":"79991458-b7db-46f4-8d3a-703762362408","Type":"ContainerDied","Data":"ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a"} Jul 08 20:01:30.115236 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:30.115218 2573 scope.go:117] "RemoveContainer" containerID="ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a" Jul 08 20:01:30.116306 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:30.116278 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:30.116407 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:30.116362 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:30.116470 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:30.116413 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:31.122682 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:31.122639 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" event={"ID":"79991458-b7db-46f4-8d3a-703762362408","Type":"ContainerStarted","Data":"07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877"} Jul 08 20:01:31.480286 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:31.480189 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:31.480473 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:31.480452 2573 patch_prober.go:28] interesting pod/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 08 20:01:31.480519 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:31.480500 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 08 20:01:32.481055 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:32.480989 2573 patch_prober.go:28] interesting pod/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 08 20:01:32.481464 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:32.481108 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 08 20:01:33.480833 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:33.480788 2573 patch_prober.go:28] interesting pod/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 08 20:01:33.481324 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:33.481289 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 08 20:01:34.481323 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:34.481286 2573 patch_prober.go:28] interesting pod/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 08 20:01:34.481722 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:34.481342 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 08 20:01:35.300679 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:35.300643 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9"] Jul 08 20:01:35.300898 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:35.300877 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" containerID="cri-o://f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f" gracePeriod=30 Jul 08 20:01:35.484972 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:35.484943 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:35.485426 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:35.485234 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:35.485937 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:35.485913 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:40.568934 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.568869 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:40.705130 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705093 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-credential-socket\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705329 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705175 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-data\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705329 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705270 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8977f8cd-05d6-4127-8f03-90271b9c588f-istiod-ca-cert\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705329 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705295 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-envoy\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705329 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705317 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-podinfo\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705556 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705339 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-token\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705556 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705400 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-certs\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705556 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705427 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzcck\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-kube-api-access-zzcck\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705556 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705408 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:40.705556 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705486 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-socket\") pod \"8977f8cd-05d6-4127-8f03-90271b9c588f\" (UID: \"8977f8cd-05d6-4127-8f03-90271b9c588f\") " Jul 08 20:01:40.705804 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705666 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:40.705804 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705709 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8977f8cd-05d6-4127-8f03-90271b9c588f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:01:40.705918 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705877 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8977f8cd-05d6-4127-8f03-90271b9c588f-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:40.705918 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705894 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:40.705918 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705907 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:40.706163 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.705991 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-data" (OuterVolumeSpecName: "istio-data") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:40.706256 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.706233 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:40.707656 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.707628 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:01:40.707767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.707706 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-token" (OuterVolumeSpecName: "istio-token") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:01:40.707959 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.707941 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:40.708042 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.707955 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-kube-api-access-zzcck" (OuterVolumeSpecName: "kube-api-access-zzcck") pod "8977f8cd-05d6-4127-8f03-90271b9c588f" (UID: "8977f8cd-05d6-4127-8f03-90271b9c588f"). InnerVolumeSpecName "kube-api-access-zzcck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:01:40.807095 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.807059 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:40.807095 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.807087 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:40.807095 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.807096 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:40.807095 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.807104 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:40.807399 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.807114 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzcck\" (UniqueName: \"kubernetes.io/projected/8977f8cd-05d6-4127-8f03-90271b9c588f-kube-api-access-zzcck\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:40.807399 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:40.807124 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8977f8cd-05d6-4127-8f03-90271b9c588f-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:41.189347 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.189312 2573 generic.go:356] "Generic (PLEG): container finished" podID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerID="f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f" exitCode=0 Jul 08 20:01:41.189549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.189358 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" Jul 08 20:01:41.189549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.189393 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" event={"ID":"8977f8cd-05d6-4127-8f03-90271b9c588f","Type":"ContainerDied","Data":"f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f"} Jul 08 20:01:41.189549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.189452 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9" event={"ID":"8977f8cd-05d6-4127-8f03-90271b9c588f","Type":"ContainerDied","Data":"ba3b4e7ecf8d68ec31d4d9551e830529a908ba8682a27a24f8cc062702c1530f"} Jul 08 20:01:41.189549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.189479 2573 scope.go:117] "RemoveContainer" containerID="f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f" Jul 08 20:01:41.199541 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.199519 2573 scope.go:117] "RemoveContainer" containerID="312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236" Jul 08 20:01:41.208752 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.208734 2573 scope.go:117] "RemoveContainer" containerID="f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f" Jul 08 20:01:41.209064 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:01:41.209043 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f\": container with ID starting with f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f not found: ID does not exist" containerID="f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f" Jul 08 20:01:41.209140 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.209075 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f"} err="failed to get container status \"f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f\": rpc error: code = NotFound desc = could not find container \"f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f\": container with ID starting with f3fa9d218c8dd8ed6c52c1e3a94effeee9567c1063b4ed5b9b5b898caa5b644f not found: ID does not exist" Jul 08 20:01:41.209140 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.209093 2573 scope.go:117] "RemoveContainer" containerID="312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236" Jul 08 20:01:41.209332 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:01:41.209317 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236\": container with ID starting with 312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236 not found: ID does not exist" containerID="312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236" Jul 08 20:01:41.209387 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.209335 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236"} err="failed to get container status \"312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236\": rpc error: code = NotFound desc = could not find container \"312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236\": container with ID starting with 312aee0fc9d09fff6b3555e43c8fc4ace2e097a37935cb407b033afd33197236 not found: ID does not exist" Jul 08 20:01:41.215992 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.215969 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9"] Jul 08 20:01:41.218333 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:41.218310 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6abdf4-openshift-default-547887db7c-8s5h9"] Jul 08 20:01:42.567572 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:42.567537 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" path="/var/lib/kubelet/pods/8977f8cd-05d6-4127-8f03-90271b9c588f/volumes" Jul 08 20:01:42.622059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:42.620084 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh"] Jul 08 20:01:42.622059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:42.620417 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" containerID="cri-o://07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877" gracePeriod=30 Jul 08 20:01:43.510012 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.509973 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2"] Jul 08 20:01:43.510464 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.510444 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" Jul 08 20:01:43.510464 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.510464 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" Jul 08 20:01:43.510605 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.510473 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" Jul 08 20:01:43.510605 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.510480 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" Jul 08 20:01:43.510605 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.510554 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" Jul 08 20:01:43.510605 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.510565 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8977f8cd-05d6-4127-8f03-90271b9c588f" containerName="istio-proxy" Jul 08 20:01:43.513808 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.513788 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.517479 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.517460 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-960620-openshift-default-dockercfg-v296r\"" Jul 08 20:01:43.526868 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.526836 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2"] Jul 08 20:01:43.634275 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634232 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-envoy\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.634275 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-podinfo\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.634845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634394 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istiod-ca-cert\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.634845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634433 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-credential-socket\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.634845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634465 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-certs\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.634845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634501 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-socket\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.634845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634559 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-data\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.634845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634576 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-token\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.634845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.634606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f866j\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-kube-api-access-f866j\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.735860 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.735823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istiod-ca-cert\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.735860 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.735861 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-credential-socket\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736164 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.735888 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-certs\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736164 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.735906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-socket\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736164 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.735941 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-data\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736164 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.735957 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-token\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736164 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.735981 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f866j\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-kube-api-access-f866j\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736164 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.735998 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-envoy\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736164 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.736012 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-podinfo\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736512 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.736342 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-credential-socket\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736512 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.736433 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-data\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736512 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.736475 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-socket\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736512 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.736478 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-certs\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.736700 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.736680 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istiod-ca-cert\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.738445 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.738415 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-envoy\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.738701 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.738674 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-podinfo\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.744736 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.744698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-token\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.744841 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.744824 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f866j\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-kube-api-access-f866j\") pod \"e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.828050 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.827939 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:43.973047 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.972991 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2"] Jul 08 20:01:43.974368 ip-10-0-143-40 kubenswrapper[2573]: W0708 20:01:43.974343 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55f87af_757f_45ce_9201_78ed4ccb1cd8.slice/crio-d3217b02138aaeaf0ab06b7ea97106f89e7da82131d2e45d55719f23dcd72b35 WatchSource:0}: Error finding container d3217b02138aaeaf0ab06b7ea97106f89e7da82131d2e45d55719f23dcd72b35: Status 404 returned error can't find the container with id d3217b02138aaeaf0ab06b7ea97106f89e7da82131d2e45d55719f23dcd72b35 Jul 08 20:01:43.976631 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.976603 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:43.976696 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.976666 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:43.976696 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:43.976693 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:44.209187 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:44.209089 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" event={"ID":"e55f87af-757f-45ce-9201-78ed4ccb1cd8","Type":"ContainerStarted","Data":"4927cb6b7fc038aa23c680c49fac0302f9913543786ae2a82ef0fe93c4992bfe"} Jul 08 20:01:44.209187 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:44.209157 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" event={"ID":"e55f87af-757f-45ce-9201-78ed4ccb1cd8","Type":"ContainerStarted","Data":"d3217b02138aaeaf0ab06b7ea97106f89e7da82131d2e45d55719f23dcd72b35"} Jul 08 20:01:44.246387 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:44.246327 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" podStartSLOduration=1.246305988 podStartE2EDuration="1.246305988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:01:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:01:44.245602023 +0000 UTC m=+2265.903040642" watchObservedRunningTime="2026-07-08 20:01:44.246305988 +0000 UTC m=+2265.903744594" Jul 08 20:01:44.829012 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:44.828968 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:45.834950 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:45.834913 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:46.220840 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:46.220752 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:46.221768 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:46.221743 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:01:47.892186 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.892119 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:47.978583 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978548 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-data\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.978794 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978593 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-credential-socket\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.978794 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978629 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79991458-b7db-46f4-8d3a-703762362408-istio-podinfo\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.978794 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978715 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-istio-token\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.978794 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978745 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj2nb\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-kube-api-access-cj2nb\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.978794 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978791 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-certs\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.979098 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978896 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-envoy\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.979098 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978906 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:47.979098 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978926 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79991458-b7db-46f4-8d3a-703762362408-istiod-ca-cert\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.979098 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.978955 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-socket\") pod \"79991458-b7db-46f4-8d3a-703762362408\" (UID: \"79991458-b7db-46f4-8d3a-703762362408\") " Jul 08 20:01:47.979319 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.979113 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:47.979319 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.979298 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:47.979451 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.979435 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:47.979507 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.979457 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:47.979507 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.979471 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:47.979593 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.979512 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-data" (OuterVolumeSpecName: "istio-data") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:47.979629 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.979612 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79991458-b7db-46f4-8d3a-703762362408-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:01:47.980970 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.980940 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-istio-token" (OuterVolumeSpecName: "istio-token") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:01:47.981117 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.980978 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-kube-api-access-cj2nb" (OuterVolumeSpecName: "kube-api-access-cj2nb") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "kube-api-access-cj2nb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:01:47.981418 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.981393 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79991458-b7db-46f4-8d3a-703762362408-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:01:47.981483 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:47.981435 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79991458-b7db-46f4-8d3a-703762362408" (UID: "79991458-b7db-46f4-8d3a-703762362408"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:01:48.080267 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.080231 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:48.080267 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.080260 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79991458-b7db-46f4-8d3a-703762362408-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:48.080267 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.080268 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79991458-b7db-46f4-8d3a-703762362408-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:48.080267 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.080276 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79991458-b7db-46f4-8d3a-703762362408-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:48.080549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.080284 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:48.080549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.080292 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj2nb\" (UniqueName: \"kubernetes.io/projected/79991458-b7db-46f4-8d3a-703762362408-kube-api-access-cj2nb\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:01:48.235773 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.235682 2573 generic.go:356] "Generic (PLEG): container finished" podID="79991458-b7db-46f4-8d3a-703762362408" containerID="07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877" exitCode=0 Jul 08 20:01:48.235773 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.235742 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" Jul 08 20:01:48.235773 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.235765 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" event={"ID":"79991458-b7db-46f4-8d3a-703762362408","Type":"ContainerDied","Data":"07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877"} Jul 08 20:01:48.236015 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.235820 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh" event={"ID":"79991458-b7db-46f4-8d3a-703762362408","Type":"ContainerDied","Data":"c6d5af672f48e758f47b287bf07ce05bfa3cfc37b148d73afc008316de4f668f"} Jul 08 20:01:48.236015 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.235844 2573 scope.go:117] "RemoveContainer" containerID="07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877" Jul 08 20:01:48.246307 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.246287 2573 scope.go:117] "RemoveContainer" containerID="ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a" Jul 08 20:01:48.254927 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.254907 2573 scope.go:117] "RemoveContainer" containerID="07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877" Jul 08 20:01:48.255262 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:01:48.255241 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877\": container with ID starting with 07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877 not found: ID does not exist" containerID="07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877" Jul 08 20:01:48.255343 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.255277 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877"} err="failed to get container status \"07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877\": rpc error: code = NotFound desc = could not find container \"07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877\": container with ID starting with 07ba21e2a915c25c30d45765e0a04657b027187a5af67aab87a03fe8157d2877 not found: ID does not exist" Jul 08 20:01:48.255343 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.255303 2573 scope.go:117] "RemoveContainer" containerID="ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a" Jul 08 20:01:48.255549 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:01:48.255531 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a\": container with ID starting with ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a not found: ID does not exist" containerID="ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a" Jul 08 20:01:48.255586 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.255555 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a"} err="failed to get container status \"ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a\": rpc error: code = NotFound desc = could not find container \"ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a\": container with ID starting with ef0e2a0be55d046a39bd346707a799b21edabd3f1b9d9c6ef828001e814e441a not found: ID does not exist" Jul 08 20:01:48.261300 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.261275 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh"] Jul 08 20:01:48.286201 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.286167 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-b7fdb0-openshift-default-54995cdbcf-cvgbh"] Jul 08 20:01:48.567600 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:48.567568 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79991458-b7db-46f4-8d3a-703762362408" path="/var/lib/kubelet/pods/79991458-b7db-46f4-8d3a-703762362408/volumes" Jul 08 20:01:51.314621 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.314589 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf"] Jul 08 20:01:51.315014 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.315004 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" Jul 08 20:01:51.315104 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.315016 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" Jul 08 20:01:51.315148 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.315119 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" Jul 08 20:01:51.315148 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.315127 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" Jul 08 20:01:51.317877 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.317860 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.320391 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.320370 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ceba08-openshift-default-dockercfg-ll8xq\"" Jul 08 20:01:51.334179 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.334144 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf"] Jul 08 20:01:51.412461 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412420 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-credential-socket\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.412461 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412464 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-certs\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.412653 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412522 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-podinfo\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.412653 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412568 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-data\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.412653 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412585 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvw2n\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-kube-api-access-pvw2n\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.412653 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-socket\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.412653 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412632 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-token\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.412836 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412660 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d194fb46-5724-4832-80ee-875b1aaf5e1d-istiod-ca-cert\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.412836 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.412679 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-envoy\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.513547 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-token\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.513729 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513555 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d194fb46-5724-4832-80ee-875b1aaf5e1d-istiod-ca-cert\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.513729 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513584 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-envoy\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.513729 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513624 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-credential-socket\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.513729 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513641 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-certs\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.513729 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513675 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-podinfo\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.513729 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513729 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-data\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.514194 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513754 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvw2n\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-kube-api-access-pvw2n\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.514194 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.513782 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-socket\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.514194 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.514099 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-credential-socket\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.514194 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.514140 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-certs\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.514194 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.514182 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-socket\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.514447 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.514215 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-data\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.514672 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.514650 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d194fb46-5724-4832-80ee-875b1aaf5e1d-istiod-ca-cert\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.515996 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.515968 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-envoy\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.516243 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.516224 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-podinfo\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.522535 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.522513 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-token\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.523092 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.523069 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvw2n\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-kube-api-access-pvw2n\") pod \"e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.629231 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.629132 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:51.805529 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.805503 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf"] Jul 08 20:01:51.808508 ip-10-0-143-40 kubenswrapper[2573]: W0708 20:01:51.808466 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd194fb46_5724_4832_80ee_875b1aaf5e1d.slice/crio-434e7675066dba0a51fe517460f17c366fd35d407c3019eba1bc2096b65cf894 WatchSource:0}: Error finding container 434e7675066dba0a51fe517460f17c366fd35d407c3019eba1bc2096b65cf894: Status 404 returned error can't find the container with id 434e7675066dba0a51fe517460f17c366fd35d407c3019eba1bc2096b65cf894 Jul 08 20:01:51.810638 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.810605 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:51.810765 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.810698 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:51.810765 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:51.810743 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:52.263400 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:52.263356 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" event={"ID":"d194fb46-5724-4832-80ee-875b1aaf5e1d","Type":"ContainerStarted","Data":"396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6"} Jul 08 20:01:52.263615 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:52.263411 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" event={"ID":"d194fb46-5724-4832-80ee-875b1aaf5e1d","Type":"ContainerStarted","Data":"434e7675066dba0a51fe517460f17c366fd35d407c3019eba1bc2096b65cf894"} Jul 08 20:01:52.291375 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:52.291319 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" podStartSLOduration=1.291300236 podStartE2EDuration="1.291300236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:01:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:01:52.289449777 +0000 UTC m=+2273.946888414" watchObservedRunningTime="2026-07-08 20:01:52.291300236 +0000 UTC m=+2273.948738841" Jul 08 20:01:52.631622 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:52.631524 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:52.645654 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:52.645617 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:53.269485 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:53.269454 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:53.270349 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:53.270329 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:01:59.305679 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:59.305640 2573 generic.go:356] "Generic (PLEG): container finished" podID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerID="4927cb6b7fc038aa23c680c49fac0302f9913543786ae2a82ef0fe93c4992bfe" exitCode=0 Jul 08 20:01:59.306143 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:59.305714 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" event={"ID":"e55f87af-757f-45ce-9201-78ed4ccb1cd8","Type":"ContainerDied","Data":"4927cb6b7fc038aa23c680c49fac0302f9913543786ae2a82ef0fe93c4992bfe"} Jul 08 20:01:59.306143 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:59.306117 2573 scope.go:117] "RemoveContainer" containerID="4927cb6b7fc038aa23c680c49fac0302f9913543786ae2a82ef0fe93c4992bfe" Jul 08 20:01:59.307154 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:59.307126 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:59.307227 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:59.307209 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:01:59.307274 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:01:59.307257 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:02:00.314731 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:00.314686 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" event={"ID":"e55f87af-757f-45ce-9201-78ed4ccb1cd8","Type":"ContainerStarted","Data":"c45312665e43bf2b1060f79337fcff756354236366257b45c6192511d3e55556"} Jul 08 20:02:00.829282 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:00.829177 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:02:00.831057 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:00.830702 2573 patch_prober.go:28] interesting pod/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 08 20:02:00.831057 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:00.830764 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 08 20:02:01.221796 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:01.221691 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:02:01.829205 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:01.829162 2573 patch_prober.go:28] interesting pod/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 08 20:02:01.829824 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:01.829786 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 08 20:02:02.828557 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:02.828516 2573 patch_prober.go:28] interesting pod/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 08 20:02:02.828838 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:02.828802 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 08 20:02:03.828539 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:03.828500 2573 patch_prober.go:28] interesting pod/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 08 20:02:03.829100 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:03.829069 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 08 20:02:04.832713 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:04.832682 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:02:04.833757 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:04.833735 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:02:06.355303 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:06.355266 2573 generic.go:356] "Generic (PLEG): container finished" podID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerID="396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6" exitCode=0 Jul 08 20:02:06.355749 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:06.355347 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" event={"ID":"d194fb46-5724-4832-80ee-875b1aaf5e1d","Type":"ContainerDied","Data":"396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6"} Jul 08 20:02:06.355884 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:06.355867 2573 scope.go:117] "RemoveContainer" containerID="396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6" Jul 08 20:02:06.357064 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:06.356996 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:02:06.357166 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:06.357086 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:02:06.357166 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:06.357120 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 08 20:02:07.366586 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:07.366550 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" event={"ID":"d194fb46-5724-4832-80ee-875b1aaf5e1d","Type":"ContainerStarted","Data":"1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421"} Jul 08 20:02:07.629967 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:07.629827 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:02:07.631221 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:07.631186 2573 patch_prober.go:28] interesting pod/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 08 20:02:07.631372 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:07.631259 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 08 20:02:08.269781 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:08.269744 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:02:08.630732 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:08.630613 2573 patch_prober.go:28] interesting pod/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 08 20:02:08.630732 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:08.630686 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 08 20:02:09.630583 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:09.630536 2573 patch_prober.go:28] interesting pod/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 08 20:02:09.630808 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:09.630611 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 08 20:02:10.634505 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:10.634468 2573 patch_prober.go:28] interesting pod/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 08 20:02:10.635148 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:10.635112 2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 08 20:02:11.634255 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:11.634221 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:02:11.635231 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:11.635208 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:02:49.986730 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:49.986680 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8"] Jul 08 20:02:49.987539 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:49.987500 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" Jul 08 20:02:49.987539 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:49.987538 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="79991458-b7db-46f4-8d3a-703762362408" containerName="istio-proxy" Jul 08 20:02:49.990106 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:49.990088 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:49.993340 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:49.993313 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ceba08\"/\"auth-test-model-ceba08-kserve-self-signed-certs\"" Jul 08 20:02:49.993488 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:49.993318 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ceba08\"/\"default-dockercfg-w2tsr\"" Jul 08 20:02:49.993488 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:49.993340 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ceba08\"/\"openshift-service-ca.crt\"" Jul 08 20:02:49.993488 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:49.993345 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ceba08\"/\"kube-root-ca.crt\"" Jul 08 20:02:50.001104 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.001074 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8"] Jul 08 20:02:50.056468 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.056427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-dshm\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.056468 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.056470 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-model-cache\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.056688 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.056600 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2227ca8b-7e31-465d-a053-6c3805be74e8-tls-certs\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.056688 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.056635 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-tmp-dir\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.056688 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.056681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-home\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.056817 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.056712 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r77qm\" (UniqueName: \"kubernetes.io/projected/2227ca8b-7e31-465d-a053-6c3805be74e8-kube-api-access-r77qm\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.157845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.157759 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r77qm\" (UniqueName: \"kubernetes.io/projected/2227ca8b-7e31-465d-a053-6c3805be74e8-kube-api-access-r77qm\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.157845 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.157820 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-dshm\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.158128 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.157872 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-model-cache\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.158128 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.157957 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2227ca8b-7e31-465d-a053-6c3805be74e8-tls-certs\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.158128 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.158063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-tmp-dir\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.158312 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.158128 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-home\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.158403 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.158382 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-model-cache\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.158454 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.158430 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-tmp-dir\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.158454 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.158442 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-home\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.160217 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.160192 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-dshm\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.160579 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.160559 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2227ca8b-7e31-465d-a053-6c3805be74e8-tls-certs\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.165898 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.165872 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r77qm\" (UniqueName: \"kubernetes.io/projected/2227ca8b-7e31-465d-a053-6c3805be74e8-kube-api-access-r77qm\") pod \"auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.300356 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.300316 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:02:50.438638 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.438605 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8"] Jul 08 20:02:50.441609 ip-10-0-143-40 kubenswrapper[2573]: W0708 20:02:50.441580 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2227ca8b_7e31_465d_a053_6c3805be74e8.slice/crio-8eb2c25d495850a3c4e57845e33b37b1abaf064956f2a04f1508b237e9c9b144 WatchSource:0}: Error finding container 8eb2c25d495850a3c4e57845e33b37b1abaf064956f2a04f1508b237e9c9b144: Status 404 returned error can't find the container with id 8eb2c25d495850a3c4e57845e33b37b1abaf064956f2a04f1508b237e9c9b144 Jul 08 20:02:50.604857 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.604746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" event={"ID":"2227ca8b-7e31-465d-a053-6c3805be74e8","Type":"ContainerStarted","Data":"1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221"} Jul 08 20:02:50.604857 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.604801 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" event={"ID":"2227ca8b-7e31-465d-a053-6c3805be74e8","Type":"ContainerStarted","Data":"8eb2c25d495850a3c4e57845e33b37b1abaf064956f2a04f1508b237e9c9b144"} Jul 08 20:02:50.634703 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:02:50.634637 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" podStartSLOduration=1.63462128 podStartE2EDuration="1.63462128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:02:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:02:50.633311923 +0000 UTC m=+2332.290750533" watchObservedRunningTime="2026-07-08 20:02:50.63462128 +0000 UTC m=+2332.292059886" Jul 08 20:03:00.300836 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:03:00.300781 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:03:00.300836 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:03:00.300853 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:03:00.313495 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:03:00.313463 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:03:00.672361 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:03:00.672267 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:04:50.499715 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:04:50.499679 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8"] Jul 08 20:05:05.519714 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.519666 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" podUID="2227ca8b-7e31-465d-a053-6c3805be74e8" containerName="main" containerID="cri-o://1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221" gracePeriod=45 Jul 08 20:05:05.772215 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.772156 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:05:05.830416 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830372 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-home\") pod \"2227ca8b-7e31-465d-a053-6c3805be74e8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " Jul 08 20:05:05.830618 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830451 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r77qm\" (UniqueName: \"kubernetes.io/projected/2227ca8b-7e31-465d-a053-6c3805be74e8-kube-api-access-r77qm\") pod \"2227ca8b-7e31-465d-a053-6c3805be74e8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " Jul 08 20:05:05.830618 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830479 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-tmp-dir\") pod \"2227ca8b-7e31-465d-a053-6c3805be74e8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " Jul 08 20:05:05.830618 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830571 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-dshm\") pod \"2227ca8b-7e31-465d-a053-6c3805be74e8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " Jul 08 20:05:05.830618 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830596 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-model-cache\") pod \"2227ca8b-7e31-465d-a053-6c3805be74e8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " Jul 08 20:05:05.830618 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830619 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2227ca8b-7e31-465d-a053-6c3805be74e8-tls-certs\") pod \"2227ca8b-7e31-465d-a053-6c3805be74e8\" (UID: \"2227ca8b-7e31-465d-a053-6c3805be74e8\") " Jul 08 20:05:05.830907 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830713 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-home" (OuterVolumeSpecName: "home") pod "2227ca8b-7e31-465d-a053-6c3805be74e8" (UID: "2227ca8b-7e31-465d-a053-6c3805be74e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:05:05.830907 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830808 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2227ca8b-7e31-465d-a053-6c3805be74e8" (UID: "2227ca8b-7e31-465d-a053-6c3805be74e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:05:05.831035 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.830990 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-home\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:05:05.831035 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.831006 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-tmp-dir\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:05:05.831182 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.831145 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "2227ca8b-7e31-465d-a053-6c3805be74e8" (UID: "2227ca8b-7e31-465d-a053-6c3805be74e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:05:05.833448 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.833414 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2227ca8b-7e31-465d-a053-6c3805be74e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2227ca8b-7e31-465d-a053-6c3805be74e8" (UID: "2227ca8b-7e31-465d-a053-6c3805be74e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 20:05:05.833571 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.833525 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-dshm" (OuterVolumeSpecName: "dshm") pod "2227ca8b-7e31-465d-a053-6c3805be74e8" (UID: "2227ca8b-7e31-465d-a053-6c3805be74e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:05:05.833648 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.833626 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2227ca8b-7e31-465d-a053-6c3805be74e8-kube-api-access-r77qm" (OuterVolumeSpecName: "kube-api-access-r77qm") pod "2227ca8b-7e31-465d-a053-6c3805be74e8" (UID: "2227ca8b-7e31-465d-a053-6c3805be74e8"). InnerVolumeSpecName "kube-api-access-r77qm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:05:05.932286 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.932246 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r77qm\" (UniqueName: \"kubernetes.io/projected/2227ca8b-7e31-465d-a053-6c3805be74e8-kube-api-access-r77qm\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:05:05.932286 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.932280 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-dshm\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:05:05.932286 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.932295 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2227ca8b-7e31-465d-a053-6c3805be74e8-model-cache\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:05:05.932286 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:05.932308 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2227ca8b-7e31-465d-a053-6c3805be74e8-tls-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:05:06.377343 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.377307 2573 generic.go:356] "Generic (PLEG): container finished" podID="2227ca8b-7e31-465d-a053-6c3805be74e8" containerID="1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221" exitCode=0 Jul 08 20:05:06.377549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.377359 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" Jul 08 20:05:06.377549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.377399 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" event={"ID":"2227ca8b-7e31-465d-a053-6c3805be74e8","Type":"ContainerDied","Data":"1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221"} Jul 08 20:05:06.377549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.377446 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8" event={"ID":"2227ca8b-7e31-465d-a053-6c3805be74e8","Type":"ContainerDied","Data":"8eb2c25d495850a3c4e57845e33b37b1abaf064956f2a04f1508b237e9c9b144"} Jul 08 20:05:06.377549 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.377465 2573 scope.go:117] "RemoveContainer" containerID="1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221" Jul 08 20:05:06.387326 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.387309 2573 scope.go:117] "RemoveContainer" containerID="1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221" Jul 08 20:05:06.387639 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:05:06.387619 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221\": container with ID starting with 1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221 not found: ID does not exist" containerID="1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221" Jul 08 20:05:06.387705 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.387650 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221"} err="failed to get container status \"1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221\": rpc error: code = NotFound desc = could not find container \"1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221\": container with ID starting with 1068859b24872e6e5a4a26ae3042460c8dcb0ee102d90258da60a67eb88c8221 not found: ID does not exist" Jul 08 20:05:06.398248 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.398213 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8"] Jul 08 20:05:06.400637 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.400612 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ceba08/auth-test-model-ceba08-kserve-6c5676f7dc-7zpq8"] Jul 08 20:05:06.568531 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:05:06.568488 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2227ca8b-7e31-465d-a053-6c3805be74e8" path="/var/lib/kubelet/pods/2227ca8b-7e31-465d-a053-6c3805be74e8/volumes" Jul 08 20:08:30.410100 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:30.410060 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2"] Jul 08 20:08:30.410606 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:30.410379 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" containerID="cri-o://c45312665e43bf2b1060f79337fcff756354236366257b45c6192511d3e55556" gracePeriod=30 Jul 08 20:08:34.836111 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:34.836069 2573 patch_prober.go:28] interesting pod/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 20:08:34.836617 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:34.836136 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 20:08:35.544141 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.544103 2573 generic.go:356] "Generic (PLEG): container finished" podID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerID="c45312665e43bf2b1060f79337fcff756354236366257b45c6192511d3e55556" exitCode=0 Jul 08 20:08:35.544298 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.544169 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" event={"ID":"e55f87af-757f-45ce-9201-78ed4ccb1cd8","Type":"ContainerDied","Data":"c45312665e43bf2b1060f79337fcff756354236366257b45c6192511d3e55556"} Jul 08 20:08:35.544298 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.544228 2573 scope.go:117] "RemoveContainer" containerID="4927cb6b7fc038aa23c680c49fac0302f9913543786ae2a82ef0fe93c4992bfe" Jul 08 20:08:35.679968 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.679900 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:08:35.798769 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.798712 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istiod-ca-cert\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.798798 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-data\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.798826 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-podinfo\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.798854 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-socket\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.798889 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-credential-socket\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.798911 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-certs\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.798957 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-envoy\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.798974 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f866j\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-kube-api-access-f866j\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799059 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.799052 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-token\") pod \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\" (UID: \"e55f87af-757f-45ce-9201-78ed4ccb1cd8\") " Jul 08 20:08:35.799500 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.799325 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:08:35.799500 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.799372 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:35.799500 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.799392 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:35.799500 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.799437 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:35.800005 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.799982 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-data" (OuterVolumeSpecName: "istio-data") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:35.801428 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.801403 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-token" (OuterVolumeSpecName: "istio-token") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:08:35.801428 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.801415 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-kube-api-access-f866j" (OuterVolumeSpecName: "kube-api-access-f866j") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "kube-api-access-f866j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:08:35.801653 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.801624 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:35.801653 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.801636 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e55f87af-757f-45ce-9201-78ed4ccb1cd8" (UID: "e55f87af-757f-45ce-9201-78ed4ccb1cd8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:08:35.900708 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900670 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:35.900708 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900700 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f866j\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-kube-api-access-f866j\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:35.900708 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900710 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:35.900708 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900720 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:35.901223 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900727 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:35.901223 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900735 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55f87af-757f-45ce-9201-78ed4ccb1cd8-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:35.901223 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900742 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:35.901223 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900749 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:35.901223 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:35.900758 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55f87af-757f-45ce-9201-78ed4ccb1cd8-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:36.551905 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:36.551874 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" Jul 08 20:08:36.552110 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:36.551903 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2" event={"ID":"e55f87af-757f-45ce-9201-78ed4ccb1cd8","Type":"ContainerDied","Data":"d3217b02138aaeaf0ab06b7ea97106f89e7da82131d2e45d55719f23dcd72b35"} Jul 08 20:08:36.552110 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:36.551956 2573 scope.go:117] "RemoveContainer" containerID="c45312665e43bf2b1060f79337fcff756354236366257b45c6192511d3e55556" Jul 08 20:08:36.573180 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:36.573146 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2"] Jul 08 20:08:36.578573 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:36.578546 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-960620-openshift-default-78cdd54c8c-z2vs2"] Jul 08 20:08:37.603521 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:37.603489 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf"] Jul 08 20:08:37.603957 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:37.603881 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" containerID="cri-o://1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421" gracePeriod=30 Jul 08 20:08:38.305358 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:38.305323 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9dfdcffcd-2dmmp_7c059129-08de-4e6f-8b2a-47a598ec4a33/authorino/0.log" Jul 08 20:08:38.568705 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:38.568623 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" path="/var/lib/kubelet/pods/e55f87af-757f-45ce-9201-78ed4ccb1cd8/volumes" Jul 08 20:08:41.637144 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:41.637107 2573 patch_prober.go:28] interesting pod/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 08 20:08:41.637542 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:41.637174 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 20:08:42.872630 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.872564 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:08:42.957447 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957406 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-data\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957447 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957451 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-token\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957698 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957472 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d194fb46-5724-4832-80ee-875b1aaf5e1d-istiod-ca-cert\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957698 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957508 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-socket\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957698 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957534 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-podinfo\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957698 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957557 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-envoy\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957698 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957620 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvw2n\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-kube-api-access-pvw2n\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957976 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957728 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-credential-socket\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957976 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957754 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-certs\") pod \"d194fb46-5724-4832-80ee-875b1aaf5e1d\" (UID: \"d194fb46-5724-4832-80ee-875b1aaf5e1d\") " Jul 08 20:08:42.957976 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957873 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:42.957976 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.957937 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d194fb46-5724-4832-80ee-875b1aaf5e1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 20:08:42.958219 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.958162 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:42.958219 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.958193 2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d194fb46-5724-4832-80ee-875b1aaf5e1d-istiod-ca-cert\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:42.958219 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.958218 2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:42.958368 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.958342 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:42.958417 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.958361 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:42.959822 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.959801 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 20:08:42.960145 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.960128 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:08:42.960212 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.960162 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 08 20:08:42.960212 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:42.960168 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-kube-api-access-pvw2n" (OuterVolumeSpecName: "kube-api-access-pvw2n") pod "d194fb46-5724-4832-80ee-875b1aaf5e1d" (UID: "d194fb46-5724-4832-80ee-875b1aaf5e1d"). InnerVolumeSpecName "kube-api-access-pvw2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 20:08:43.058692 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.058642 2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-credential-socket\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:43.058692 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.058689 2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-workload-certs\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:43.058692 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.058703 2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-data\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:43.058939 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.058714 2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-token\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:43.058939 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.058724 2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-podinfo\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:43.058939 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.058734 2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d194fb46-5724-4832-80ee-875b1aaf5e1d-istio-envoy\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:43.058939 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.058745 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvw2n\" (UniqueName: \"kubernetes.io/projected/d194fb46-5724-4832-80ee-875b1aaf5e1d-kube-api-access-pvw2n\") on node \"ip-10-0-143-40.ec2.internal\" DevicePath \"\"" Jul 08 20:08:43.088618 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.088587 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bkcpr_39a574bc-8799-4a4a-b869-21843ab70b28/manager/0.log" Jul 08 20:08:43.224510 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.224429 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9d6v6_c71dc895-238f-48a9-9b32-b672b43ba401/manager/0.log" Jul 08 20:08:43.593948 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.593914 2573 generic.go:356] "Generic (PLEG): container finished" podID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerID="1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421" exitCode=0 Jul 08 20:08:43.594171 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.593981 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" Jul 08 20:08:43.594171 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.594001 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" event={"ID":"d194fb46-5724-4832-80ee-875b1aaf5e1d","Type":"ContainerDied","Data":"1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421"} Jul 08 20:08:43.594171 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.594080 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf" event={"ID":"d194fb46-5724-4832-80ee-875b1aaf5e1d","Type":"ContainerDied","Data":"434e7675066dba0a51fe517460f17c366fd35d407c3019eba1bc2096b65cf894"} Jul 08 20:08:43.594171 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.594104 2573 scope.go:117] "RemoveContainer" containerID="1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421" Jul 08 20:08:43.604231 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.604211 2573 scope.go:117] "RemoveContainer" containerID="396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6" Jul 08 20:08:43.613149 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.613128 2573 scope.go:117] "RemoveContainer" containerID="1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421" Jul 08 20:08:43.613489 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:08:43.613455 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421\": container with ID starting with 1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421 not found: ID does not exist" containerID="1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421" Jul 08 20:08:43.613617 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.613496 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421"} err="failed to get container status \"1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421\": rpc error: code = NotFound desc = could not find container \"1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421\": container with ID starting with 1fffcde3d651eeec3778b5a245a0b9ccddcdc0fba1fe49db5eab7eb7925b8421 not found: ID does not exist" Jul 08 20:08:43.613617 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.613522 2573 scope.go:117] "RemoveContainer" containerID="396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6" Jul 08 20:08:43.613865 ip-10-0-143-40 kubenswrapper[2573]: E0708 20:08:43.613844 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6\": container with ID starting with 396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6 not found: ID does not exist" containerID="396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6" Jul 08 20:08:43.613948 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.613915 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6"} err="failed to get container status \"396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6\": rpc error: code = NotFound desc = could not find container \"396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6\": container with ID starting with 396ff134133e3feea3fb862ab5767d98bbb6e5116fc23656c9c748d18b50eec6 not found: ID does not exist" Jul 08 20:08:43.616514 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.616492 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf"] Jul 08 20:08:43.622871 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.622846 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ceba08-openshift-default-85db8d559f-sm6bf"] Jul 08 20:08:43.975528 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:43.975429 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nlj55_f241326d-2cef-4908-9037-c58572d8ff09/manager/0.log" Jul 08 20:08:44.244579 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:44.244556 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77ddc6fd87-kvrlk_ea1c2768-c0e8-4e42-89a1-37f2298434e6/copy-manifests/0.log" Jul 08 20:08:44.256755 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:44.256732 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77ddc6fd87-kvrlk_ea1c2768-c0e8-4e42-89a1-37f2298434e6/manager/0.log" Jul 08 20:08:44.567838 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:44.567752 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" path="/var/lib/kubelet/pods/d194fb46-5724-4832-80ee-875b1aaf5e1d/volumes" Jul 08 20:08:45.293675 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.293651 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l_b47e78cb-e83e-46a8-b391-d15363b90f9f/util/0.log" Jul 08 20:08:45.300417 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.300396 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l_b47e78cb-e83e-46a8-b391-d15363b90f9f/pull/0.log" Jul 08 20:08:45.307223 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.307181 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l_b47e78cb-e83e-46a8-b391-d15363b90f9f/extract/0.log" Jul 08 20:08:45.428187 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.428160 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25_1b89c29e-1808-453d-8cbe-f356a0498d7d/util/0.log" Jul 08 20:08:45.434049 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.434015 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25_1b89c29e-1808-453d-8cbe-f356a0498d7d/pull/0.log" Jul 08 20:08:45.440034 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.440006 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25_1b89c29e-1808-453d-8cbe-f356a0498d7d/extract/0.log" Jul 08 20:08:45.576207 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.576133 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w_30374b44-9297-43c7-b286-71ec0955811f/util/0.log" Jul 08 20:08:45.583008 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.582983 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w_30374b44-9297-43c7-b286-71ec0955811f/pull/0.log" Jul 08 20:08:45.589328 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.589306 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w_30374b44-9297-43c7-b286-71ec0955811f/extract/0.log" Jul 08 20:08:45.708138 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.708113 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p_cbfe2a9d-2ab8-4df8-b52d-a36fc2616889/pull/0.log" Jul 08 20:08:45.714257 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.714236 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p_cbfe2a9d-2ab8-4df8-b52d-a36fc2616889/extract/0.log" Jul 08 20:08:45.719993 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.719976 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p_cbfe2a9d-2ab8-4df8-b52d-a36fc2616889/util/0.log" Jul 08 20:08:45.841379 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.841274 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9dfdcffcd-2dmmp_7c059129-08de-4e6f-8b2a-47a598ec4a33/authorino/0.log" Jul 08 20:08:45.966833 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:45.966806 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ttps6_0f2532d9-bf2c-48e7-877e-c3384f674cc4/manager/0.log" Jul 08 20:08:46.084952 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:46.084911 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7kjk9_6b9542bb-3547-4c88-b926-5445a3279e34/manager/0.log" Jul 08 20:08:46.315436 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:46.315391 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2x66b_c16e45b4-71de-40a7-8338-201c01495f66/registry-server/0.log" Jul 08 20:08:46.440306 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:46.440272 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gtkqt_6b75d7dc-7b27-408f-a9c3-1d968c071597/manager/0.log" Jul 08 20:08:46.694863 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:46.694772 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h92wd_2009d245-c529-4f6b-96a4-f02cf48b7904/manager/0.log" Jul 08 20:08:47.052008 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:47.051980 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mzqnp_551885ec-5d6d-431e-af1c-7318fcf75bcc/istio-proxy/0.log" Jul 08 20:08:47.303685 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:47.303600 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b84bffb4-djnqq_5674a395-0063-49c9-8316-3c2302623d8b/kube-auth-proxy/0.log" Jul 08 20:08:47.556245 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:47.556134 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6rz4f_0ed74d63-b86f-4bc3-a7ef-fe075ab472dd/istio-proxy/1.log" Jul 08 20:08:48.427905 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:48.427861 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-2h8ds_f58668d1-15c8-42a4-a316-ca66cc7d367b/main/0.log" Jul 08 20:08:48.664168 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:48.664133 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db7lzc4_612c66e9-602f-446a-9639-d13c7122e332/main/0.log" Jul 08 20:08:53.532410 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532332 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4cwpq/must-gather-gzdqb"] Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532695 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532709 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532722 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532727 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532734 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532739 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532745 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2227ca8b-7e31-465d-a053-6c3805be74e8" containerName="main" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532750 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2227ca8b-7e31-465d-a053-6c3805be74e8" containerName="main" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532760 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" Jul 08 20:08:53.532767 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532765 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" Jul 08 20:08:53.533145 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532824 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" Jul 08 20:08:53.533145 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532832 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d194fb46-5724-4832-80ee-875b1aaf5e1d" containerName="istio-proxy" Jul 08 20:08:53.533145 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532837 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2227ca8b-7e31-465d-a053-6c3805be74e8" containerName="main" Jul 08 20:08:53.533145 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532844 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" Jul 08 20:08:53.533145 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.532946 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e55f87af-757f-45ce-9201-78ed4ccb1cd8" containerName="istio-proxy" Jul 08 20:08:53.536268 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.536252 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cwpq/must-gather-gzdqb" Jul 08 20:08:53.538759 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.538721 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4cwpq\"/\"kube-root-ca.crt\"" Jul 08 20:08:53.538865 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.538763 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4cwpq\"/\"default-dockercfg-jzmnh\"" Jul 08 20:08:53.539372 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.539358 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4cwpq\"/\"openshift-service-ca.crt\"" Jul 08 20:08:53.544722 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.544698 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4cwpq/must-gather-gzdqb"] Jul 08 20:08:53.643688 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.643647 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bwn6\" (UniqueName: \"kubernetes.io/projected/fd04ff78-50e9-42e3-905b-1ef56bbe0900-kube-api-access-9bwn6\") pod \"must-gather-gzdqb\" (UID: \"fd04ff78-50e9-42e3-905b-1ef56bbe0900\") " pod="openshift-must-gather-4cwpq/must-gather-gzdqb" Jul 08 20:08:53.643961 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.643741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd04ff78-50e9-42e3-905b-1ef56bbe0900-must-gather-output\") pod \"must-gather-gzdqb\" (UID: \"fd04ff78-50e9-42e3-905b-1ef56bbe0900\") " pod="openshift-must-gather-4cwpq/must-gather-gzdqb" Jul 08 20:08:53.744989 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.744952 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd04ff78-50e9-42e3-905b-1ef56bbe0900-must-gather-output\") pod \"must-gather-gzdqb\" (UID: \"fd04ff78-50e9-42e3-905b-1ef56bbe0900\") " pod="openshift-must-gather-4cwpq/must-gather-gzdqb" Jul 08 20:08:53.745191 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.745017 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bwn6\" (UniqueName: \"kubernetes.io/projected/fd04ff78-50e9-42e3-905b-1ef56bbe0900-kube-api-access-9bwn6\") pod \"must-gather-gzdqb\" (UID: \"fd04ff78-50e9-42e3-905b-1ef56bbe0900\") " pod="openshift-must-gather-4cwpq/must-gather-gzdqb" Jul 08 20:08:53.745348 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.745326 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fd04ff78-50e9-42e3-905b-1ef56bbe0900-must-gather-output\") pod \"must-gather-gzdqb\" (UID: \"fd04ff78-50e9-42e3-905b-1ef56bbe0900\") " pod="openshift-must-gather-4cwpq/must-gather-gzdqb" Jul 08 20:08:53.753101 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.753073 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bwn6\" (UniqueName: \"kubernetes.io/projected/fd04ff78-50e9-42e3-905b-1ef56bbe0900-kube-api-access-9bwn6\") pod \"must-gather-gzdqb\" (UID: \"fd04ff78-50e9-42e3-905b-1ef56bbe0900\") " pod="openshift-must-gather-4cwpq/must-gather-gzdqb" Jul 08 20:08:53.846661 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.846576 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cwpq/must-gather-gzdqb" Jul 08 20:08:53.986640 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.986612 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4cwpq/must-gather-gzdqb"] Jul 08 20:08:53.988860 ip-10-0-143-40 kubenswrapper[2573]: W0708 20:08:53.988823 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd04ff78_50e9_42e3_905b_1ef56bbe0900.slice/crio-bbd68d1a4477991df6198e6b7097da3f4f7b6f8e7a033c816bba499b74671284 WatchSource:0}: Error finding container bbd68d1a4477991df6198e6b7097da3f4f7b6f8e7a033c816bba499b74671284: Status 404 returned error can't find the container with id bbd68d1a4477991df6198e6b7097da3f4f7b6f8e7a033c816bba499b74671284 Jul 08 20:08:53.990744 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:53.990727 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 20:08:54.654370 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:54.654327 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cwpq/must-gather-gzdqb" event={"ID":"fd04ff78-50e9-42e3-905b-1ef56bbe0900","Type":"ContainerStarted","Data":"bbd68d1a4477991df6198e6b7097da3f4f7b6f8e7a033c816bba499b74671284"} Jul 08 20:08:55.666508 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:55.666418 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cwpq/must-gather-gzdqb" event={"ID":"fd04ff78-50e9-42e3-905b-1ef56bbe0900","Type":"ContainerStarted","Data":"e2fe2a6f5bb6abcd41494eaea5f5808760c49c496a9ae06368b9c2c2fa89282a"} Jul 08 20:08:55.666508 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:55.666473 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cwpq/must-gather-gzdqb" event={"ID":"fd04ff78-50e9-42e3-905b-1ef56bbe0900","Type":"ContainerStarted","Data":"6f3506d0da676d585d9350fde9de4bb3c63e6f0ed649bbd6f68daa7426da5f35"} Jul 08 20:08:55.683359 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:55.683293 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4cwpq/must-gather-gzdqb" podStartSLOduration=1.417773152 podStartE2EDuration="2.683271562s" totalImagesPullingTime="1.26549841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 20:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:08:55.680938819 +0000 UTC m=+2697.338377426" watchObservedRunningTime="2026-07-08 20:08:55.683271562 +0000 UTC m=+2697.340710170" Jul 08 20:08:56.842776 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:56.842736 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ccpl9_38604f08-ccce-444b-a72f-568e18ce44c9/global-pull-secret-syncer/0.log" Jul 08 20:08:57.006190 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:57.006157 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nbsqd_727ac1df-ef0a-459c-ac26-fffd4553bc86/konnectivity-agent/0.log" Jul 08 20:08:57.117312 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:08:57.117237 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-40.ec2.internal_02f3c755db773470cb39b7721b58aba0/haproxy/0.log" Jul 08 20:09:01.514706 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.514670 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l_b47e78cb-e83e-46a8-b391-d15363b90f9f/extract/0.log" Jul 08 20:09:01.541730 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.541704 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l_b47e78cb-e83e-46a8-b391-d15363b90f9f/util/0.log" Jul 08 20:09:01.567342 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.567306 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759sr62l_b47e78cb-e83e-46a8-b391-d15363b90f9f/pull/0.log" Jul 08 20:09:01.599992 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.599940 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25_1b89c29e-1808-453d-8cbe-f356a0498d7d/extract/0.log" Jul 08 20:09:01.624931 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.624905 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25_1b89c29e-1808-453d-8cbe-f356a0498d7d/util/0.log" Jul 08 20:09:01.645821 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.645795 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08kr25_1b89c29e-1808-453d-8cbe-f356a0498d7d/pull/0.log" Jul 08 20:09:01.674510 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.674482 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w_30374b44-9297-43c7-b286-71ec0955811f/extract/0.log" Jul 08 20:09:01.697075 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.697049 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w_30374b44-9297-43c7-b286-71ec0955811f/util/0.log" Jul 08 20:09:01.723858 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.723824 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tq84w_30374b44-9297-43c7-b286-71ec0955811f/pull/0.log" Jul 08 20:09:01.752687 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.752645 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p_cbfe2a9d-2ab8-4df8-b52d-a36fc2616889/extract/0.log" Jul 08 20:09:01.776578 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.776492 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p_cbfe2a9d-2ab8-4df8-b52d-a36fc2616889/util/0.log" Jul 08 20:09:01.798114 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:01.798084 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef12cd5p_cbfe2a9d-2ab8-4df8-b52d-a36fc2616889/pull/0.log" Jul 08 20:09:02.213093 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:02.212259 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-9dfdcffcd-2dmmp_7c059129-08de-4e6f-8b2a-47a598ec4a33/authorino/0.log" Jul 08 20:09:02.247854 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:02.247817 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-ttps6_0f2532d9-bf2c-48e7-877e-c3384f674cc4/manager/0.log" Jul 08 20:09:02.283068 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:02.282950 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7kjk9_6b9542bb-3547-4c88-b926-5445a3279e34/manager/0.log" Jul 08 20:09:02.365848 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:02.365811 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2x66b_c16e45b4-71de-40a7-8338-201c01495f66/registry-server/0.log" Jul 08 20:09:02.553287 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:02.550885 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gtkqt_6b75d7dc-7b27-408f-a9c3-1d968c071597/manager/0.log" Jul 08 20:09:02.666189 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:02.666093 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h92wd_2009d245-c529-4f6b-96a4-f02cf48b7904/manager/0.log" Jul 08 20:09:04.540118 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:04.540091 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-knm4n_c947590c-8931-41a1-b313-183c2d23d223/monitoring-plugin/0.log" Jul 08 20:09:04.814432 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:04.814341 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mtpkd_3615a4c9-ae12-409e-ac2d-bb3a94eba7d0/node-exporter/0.log" Jul 08 20:09:04.843502 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:04.843469 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mtpkd_3615a4c9-ae12-409e-ac2d-bb3a94eba7d0/kube-rbac-proxy/0.log" Jul 08 20:09:04.871694 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:04.871657 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mtpkd_3615a4c9-ae12-409e-ac2d-bb3a94eba7d0/init-textfile/0.log" Jul 08 20:09:05.260192 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:05.260150 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b5945578-mqtsv_ce16abff-9081-4b84-9614-16efb6927f98/telemeter-client/0.log" Jul 08 20:09:05.289951 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:05.289923 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b5945578-mqtsv_ce16abff-9081-4b84-9614-16efb6927f98/reload/0.log" Jul 08 20:09:05.312241 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:05.312207 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b5945578-mqtsv_ce16abff-9081-4b84-9614-16efb6927f98/kube-rbac-proxy/0.log" Jul 08 20:09:07.501756 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.501701 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4"] Jul 08 20:09:07.506881 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.506855 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.514012 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.513984 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4"] Jul 08 20:09:07.614444 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.614411 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-podres\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.614731 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.614710 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-sys\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.614871 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.614856 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-lib-modules\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.615292 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.615272 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-proc\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.615489 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.615474 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq56v\" (UniqueName: \"kubernetes.io/projected/5c7ea645-f49a-4064-bc63-2dd62452bb23-kube-api-access-qq56v\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.716629 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.716589 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq56v\" (UniqueName: \"kubernetes.io/projected/5c7ea645-f49a-4064-bc63-2dd62452bb23-kube-api-access-qq56v\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.716829 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.716710 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-podres\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.716829 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.716769 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-sys\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.716829 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.716795 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-lib-modules\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.716829 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.716826 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-proc\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.717110 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.716968 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-proc\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.717466 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.717431 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-podres\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.717545 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.717511 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-sys\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.717617 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.717599 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c7ea645-f49a-4064-bc63-2dd62452bb23-lib-modules\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.727906 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.727878 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq56v\" (UniqueName: \"kubernetes.io/projected/5c7ea645-f49a-4064-bc63-2dd62452bb23-kube-api-access-qq56v\") pod \"perf-node-gather-daemonset-56np4\" (UID: \"5c7ea645-f49a-4064-bc63-2dd62452bb23\") " pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.806294 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.806217 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-789756545c-f752n_68106ee9-eb9f-46ca-bc04-44371467d9cc/console/0.log" Jul 08 20:09:07.822813 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.822785 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:07.842766 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:07.842740 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-8p4w4_d0c477c4-df46-4dc0-9a1c-dceeeb0fb04a/download-server/0.log" Jul 08 20:09:08.007178 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:08.007150 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4"] Jul 08 20:09:08.009507 ip-10-0-143-40 kubenswrapper[2573]: W0708 20:09:08.009359 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5c7ea645_f49a_4064_bc63_2dd62452bb23.slice/crio-dd09cc0a03c942ee9d873b026bba4ed6b8a03ecff160d1b6220cf8ec556f349a WatchSource:0}: Error finding container dd09cc0a03c942ee9d873b026bba4ed6b8a03ecff160d1b6220cf8ec556f349a: Status 404 returned error can't find the container with id dd09cc0a03c942ee9d873b026bba4ed6b8a03ecff160d1b6220cf8ec556f349a Jul 08 20:09:08.782793 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:08.782687 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" event={"ID":"5c7ea645-f49a-4064-bc63-2dd62452bb23","Type":"ContainerStarted","Data":"333e2848934d6f30a6aa49eb9c74fdb3727576deec5b07e8ffe56004059a1a76"} Jul 08 20:09:08.782793 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:08.782761 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" event={"ID":"5c7ea645-f49a-4064-bc63-2dd62452bb23","Type":"ContainerStarted","Data":"dd09cc0a03c942ee9d873b026bba4ed6b8a03ecff160d1b6220cf8ec556f349a"} Jul 08 20:09:08.783599 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:08.783578 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:08.801559 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:08.801496 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" podStartSLOduration=1.801476814 podStartE2EDuration="1.801476814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 20:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 20:09:08.799400937 +0000 UTC m=+2710.456839759" watchObservedRunningTime="2026-07-08 20:09:08.801476814 +0000 UTC m=+2710.458915424" Jul 08 20:09:09.533008 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:09.532977 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gs282_ab86e6aa-3980-4d63-b431-1b0bfe2caadd/dns/0.log" Jul 08 20:09:09.554830 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:09.554805 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gs282_ab86e6aa-3980-4d63-b431-1b0bfe2caadd/kube-rbac-proxy/0.log" Jul 08 20:09:09.666472 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:09.666438 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s9zdv_3c38ec85-addf-4b25-b679-ba6f3c049625/dns-node-resolver/0.log" Jul 08 20:09:10.258805 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:10.258768 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rwqhn_2d3cd6ed-e75b-411c-9e7d-5e23bf9e9542/node-ca/0.log" Jul 08 20:09:11.247744 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:11.247716 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mzqnp_551885ec-5d6d-431e-af1c-7318fcf75bcc/istio-proxy/0.log" Jul 08 20:09:11.452241 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:11.452217 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b84bffb4-djnqq_5674a395-0063-49c9-8316-3c2302623d8b/kube-auth-proxy/0.log" Jul 08 20:09:11.510146 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:11.510071 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6rz4f_0ed74d63-b86f-4bc3-a7ef-fe075ab472dd/istio-proxy/0.log" Jul 08 20:09:11.548756 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:11.548710 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6rz4f_0ed74d63-b86f-4bc3-a7ef-fe075ab472dd/istio-proxy/1.log" Jul 08 20:09:12.414433 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:12.414397 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6x7mh_bbad0080-35fb-42b3-883d-f81cb83f7789/serve-healthcheck-canary/0.log" Jul 08 20:09:12.922139 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:12.922114 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2stkk_45c85443-2edc-4833-bc7e-bb7efb5861d7/kube-rbac-proxy/0.log" Jul 08 20:09:12.943304 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:12.943279 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2stkk_45c85443-2edc-4833-bc7e-bb7efb5861d7/exporter/0.log" Jul 08 20:09:12.967237 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:12.967208 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2stkk_45c85443-2edc-4833-bc7e-bb7efb5861d7/extractor/0.log" Jul 08 20:09:15.186856 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:15.186827 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-bkcpr_39a574bc-8799-4a4a-b869-21843ab70b28/manager/0.log" Jul 08 20:09:15.359849 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:15.359810 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9d6v6_c71dc895-238f-48a9-9b32-b672b43ba401/manager/0.log" Jul 08 20:09:15.704009 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:15.703971 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-nlj55_f241326d-2cef-4908-9037-c58572d8ff09/manager/0.log" Jul 08 20:09:15.810560 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:15.810534 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4cwpq/perf-node-gather-daemonset-56np4" Jul 08 20:09:15.838854 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:15.838821 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77ddc6fd87-kvrlk_ea1c2768-c0e8-4e42-89a1-37f2298434e6/manager/0.log" Jul 08 20:09:15.856794 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:15.856763 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-77ddc6fd87-kvrlk_ea1c2768-c0e8-4e42-89a1-37f2298434e6/copy-manifests/0.log" Jul 08 20:09:17.325401 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:17.325356 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-sgr4q_217f7efe-cd01-4b57-8e98-3d52d6420125/openshift-lws-operator/0.log" Jul 08 20:09:17.743779 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:17.743750 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mzqnp_551885ec-5d6d-431e-af1c-7318fcf75bcc/istio-proxy/0.log" Jul 08 20:09:17.947449 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:17.947421 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b84bffb4-djnqq_5674a395-0063-49c9-8316-3c2302623d8b/kube-auth-proxy/0.log" Jul 08 20:09:17.985853 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:17.985821 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6rz4f_0ed74d63-b86f-4bc3-a7ef-fe075ab472dd/istio-proxy/0.log" Jul 08 20:09:18.047251 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:18.047218 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6rz4f_0ed74d63-b86f-4bc3-a7ef-fe075ab472dd/istio-proxy/1.log" Jul 08 20:09:19.349425 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:19.349396 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mzqnp_551885ec-5d6d-431e-af1c-7318fcf75bcc/istio-proxy/0.log" Jul 08 20:09:19.541071 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:19.541039 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b84bffb4-djnqq_5674a395-0063-49c9-8316-3c2302623d8b/kube-auth-proxy/0.log" Jul 08 20:09:19.580574 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:19.580544 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6rz4f_0ed74d63-b86f-4bc3-a7ef-fe075ab472dd/istio-proxy/0.log" Jul 08 20:09:19.620715 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:19.620603 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-6rz4f_0ed74d63-b86f-4bc3-a7ef-fe075ab472dd/istio-proxy/1.log" Jul 08 20:09:24.429565 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.429534 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-svsl5_5c41a986-2d51-4475-af63-c8f18729ccdf/kube-multus-additional-cni-plugins/0.log" Jul 08 20:09:24.450588 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.450559 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-svsl5_5c41a986-2d51-4475-af63-c8f18729ccdf/egress-router-binary-copy/0.log" Jul 08 20:09:24.475593 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.475567 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-svsl5_5c41a986-2d51-4475-af63-c8f18729ccdf/cni-plugins/0.log" Jul 08 20:09:24.495818 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.495793 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-svsl5_5c41a986-2d51-4475-af63-c8f18729ccdf/bond-cni-plugin/0.log" Jul 08 20:09:24.516367 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.516339 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-svsl5_5c41a986-2d51-4475-af63-c8f18729ccdf/routeoverride-cni/0.log" Jul 08 20:09:24.536054 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.536008 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-svsl5_5c41a986-2d51-4475-af63-c8f18729ccdf/whereabouts-cni-bincopy/0.log" Jul 08 20:09:24.556372 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.556345 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-svsl5_5c41a986-2d51-4475-af63-c8f18729ccdf/whereabouts-cni/0.log" Jul 08 20:09:24.629069 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.629006 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h2hxc_a57acc89-7b79-46d7-9483-40eed3dac74f/kube-multus/0.log" Jul 08 20:09:24.685055 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.684953 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ndsk6_582ae813-01ce-4fef-86ea-6dfc6ea19f3d/network-metrics-daemon/0.log" Jul 08 20:09:24.704740 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:24.704697 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ndsk6_582ae813-01ce-4fef-86ea-6dfc6ea19f3d/kube-rbac-proxy/0.log" Jul 08 20:09:25.894012 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:25.893983 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgfrh_3d991d98-b53e-49b8-9801-9871673c42cf/ovn-controller/0.log" Jul 08 20:09:25.914267 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:25.914234 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgfrh_3d991d98-b53e-49b8-9801-9871673c42cf/ovn-acl-logging/0.log" Jul 08 20:09:25.946552 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:25.946508 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgfrh_3d991d98-b53e-49b8-9801-9871673c42cf/kube-rbac-proxy-node/0.log" Jul 08 20:09:25.973925 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:25.973894 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgfrh_3d991d98-b53e-49b8-9801-9871673c42cf/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 20:09:25.995710 ip-10-0-143-40 kubenswrapper[2573]: I0708 20:09:25.995681 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgfrh_3d991d98-b53e-49b8-9801-9871673c42cf/northd/0.log"