Jul 13 17:17:41.737623 ip-10-0-143-131 systemd[1]: Starting Kubernetes Kubelet... Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.099956 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107031 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107072 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107084 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107087 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107092 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107222 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107225 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:17:42.121906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107228 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107231 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107234 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107236 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107239 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107242 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107244 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107247 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107251 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107255 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107258 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107261 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107263 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107267 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107271 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107273 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107276 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107279 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:17:42.124148 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107281 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107285 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107288 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107291 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107293 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107296 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107300 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107303 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107305 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107308 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107315 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107333 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107337 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107339 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107343 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107346 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107349 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107352 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107354 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:17:42.126649 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107357 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107360 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107362 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107365 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107369 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107372 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107374 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107377 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107380 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107382 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107385 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107388 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107391 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107393 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107396 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107398 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107401 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107403 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107406 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107409 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:17:42.127554 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107412 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107414 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107417 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107420 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107422 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107426 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107429 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107432 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107434 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107437 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107440 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107442 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107445 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107447 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107450 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107452 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107455 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107457 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107460 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107463 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:17:42.128730 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107466 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107468 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107471 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107475 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107477 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107480 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107482 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107485 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107487 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107490 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107493 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107495 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107498 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107500 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.107503 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108180 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108186 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108190 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108192 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108195 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:17:42.132188 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108197 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108200 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108203 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108206 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108208 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108211 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108214 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108217 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108219 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108222 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108224 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108227 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108231 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108233 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108236 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108239 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108244 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108246 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108249 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108252 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:17:42.132891 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108254 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108256 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108259 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108261 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108265 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108269 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108272 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108275 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108278 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108281 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108284 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108286 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108289 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108292 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108295 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108298 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108301 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108303 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108306 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108308 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:17:42.148100 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108311 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108313 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108316 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108336 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108338 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108341 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108344 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108347 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108352 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108355 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108358 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108361 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108363 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108366 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108369 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108371 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108374 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108377 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108379 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108382 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:17:42.149011 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108385 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108390 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108394 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108396 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108401 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108403 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108406 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108408 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108412 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108415 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108418 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108421 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108423 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108426 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108429 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108432 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108435 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108437 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108441 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:17:42.150883 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108443 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108447 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108450 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108452 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108454 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108457 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108459 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108462 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108464 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108467 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108469 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108472 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108474 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108476 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108479 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108830 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108834 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108839 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108841 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108844 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:17:42.152488 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108846 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108849 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108851 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108854 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108856 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108860 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108864 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108867 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108870 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108873 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108875 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108878 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108880 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108884 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108887 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108889 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108892 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108894 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108897 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108899 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:17:42.153741 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108902 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108904 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108907 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108909 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108911 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108914 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108916 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108919 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108922 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108927 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108931 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108933 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108936 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108939 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108941 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108944 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108946 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108948 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108951 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108953 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:17:42.154692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108956 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108958 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108960 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108963 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108966 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108969 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108972 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108975 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108977 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108980 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108983 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108985 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108987 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108990 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108993 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108995 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.108997 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109000 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109002 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109004 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:17:42.155432 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109007 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109011 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109014 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109016 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109018 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109021 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109023 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109026 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109028 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109031 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109033 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109035 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109038 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109040 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109043 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109046 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109050 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109055 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109058 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:17:42.156104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109060 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109063 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109065 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109068 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109070 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109072 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109075 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109077 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109079 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109082 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109085 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109087 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109090 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109092 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109096 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109165 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109172 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109178 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109191 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109195 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109198 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 17:17:42.156809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109201 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109207 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109210 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109213 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109217 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109220 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109223 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109226 2579 flags.go:64] FLAG: --cgroup-root="" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109229 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109234 2579 flags.go:64] FLAG: --client-ca-file="" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109238 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109241 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109245 2579 flags.go:64] FLAG: --cluster-domain="" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109248 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109251 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109254 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109258 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109261 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109264 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109267 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109270 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109273 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109276 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109279 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 17:17:42.157521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109282 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109286 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109289 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109292 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109295 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109298 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109300 2579 flags.go:64] FLAG: --enable-server="true" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109303 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109307 2579 flags.go:64] FLAG: --event-burst="100" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109310 2579 flags.go:64] FLAG: --event-qps="50" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109313 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109316 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109333 2579 flags.go:64] FLAG: --eviction-hard="" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109336 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109339 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109342 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109345 2579 flags.go:64] FLAG: --eviction-soft="" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109349 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109352 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109356 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109359 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109362 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109364 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109367 2579 flags.go:64] FLAG: --feature-gates="" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109371 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 17:17:42.158217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109374 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109377 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109380 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109382 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109385 2579 flags.go:64] FLAG: --help="false" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109388 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109391 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109394 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109397 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109401 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109404 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109406 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109409 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109412 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109415 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109417 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109420 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109423 2579 flags.go:64] FLAG: --kube-reserved="" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109426 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109429 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109432 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109434 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109437 2579 flags.go:64] FLAG: --lock-file="" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109439 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 17:17:42.159174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109445 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109447 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109452 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109456 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109459 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109462 2579 flags.go:64] FLAG: --logging-format="text" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109465 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109468 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109470 2579 flags.go:64] FLAG: --manifest-url="" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109473 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109477 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109480 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109483 2579 flags.go:64] FLAG: --max-pods="110" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109486 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109489 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109491 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109496 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109499 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109502 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109504 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109511 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109514 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109517 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 17:17:42.159821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109519 2579 flags.go:64] FLAG: --pod-cidr="" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109522 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109528 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109531 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109534 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109537 2579 flags.go:64] FLAG: --port="10250" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109540 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109543 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bcb1450d08ceee0f" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109546 2579 flags.go:64] FLAG: --qos-reserved="" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109549 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109553 2579 flags.go:64] FLAG: --register-node="true" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109555 2579 flags.go:64] FLAG: --register-with-taints="" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109559 2579 flags.go:64] FLAG: --registry-burst="10" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109562 2579 flags.go:64] FLAG: --registry-qps="5" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109565 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109568 2579 flags.go:64] FLAG: --reserved-memory="" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109571 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109574 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109577 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109580 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109583 2579 flags.go:64] FLAG: --runonce="false" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109586 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109589 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109591 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109594 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 17:17:42.160527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109598 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109601 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109604 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109607 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109610 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109613 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109615 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109618 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109621 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109624 2579 flags.go:64] FLAG: --system-cgroups="" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109626 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109631 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109634 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109637 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109641 2579 flags.go:64] FLAG: --tls-min-version="" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109644 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109646 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109650 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109653 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109656 2579 flags.go:64] FLAG: --v="2" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109659 2579 flags.go:64] FLAG: --version="false" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109664 2579 flags.go:64] FLAG: --vmodule="" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109667 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.109671 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109788 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:17:42.161187 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109791 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109794 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109797 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109799 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109802 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109804 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109807 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109811 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109815 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109818 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109821 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109823 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109826 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109828 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109831 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109833 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109836 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109838 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109840 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:17:42.161881 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109843 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109845 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109848 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109850 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109853 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109856 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109859 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109861 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109864 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109866 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109869 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109872 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109874 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109877 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109879 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109881 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109884 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109886 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109889 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:17:42.162616 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109891 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109902 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109905 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109907 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109910 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109912 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109915 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109918 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109920 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109923 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109925 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109928 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109948 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109951 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109954 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109956 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109959 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109961 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109965 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109968 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:17:42.163334 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109970 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109973 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109975 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109978 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109982 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109984 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109986 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109989 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109992 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109994 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109997 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.109999 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110002 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110005 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110008 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110010 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110013 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110016 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110018 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110021 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:17:42.164024 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110023 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110026 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110028 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110031 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110034 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110036 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110038 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110041 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110043 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110046 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110049 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110052 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110055 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110059 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110062 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110065 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110069 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110073 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110076 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:17:42.164669 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.110079 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.110702 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.117199 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.117213 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.117229 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.117238 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117292 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117297 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117301 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117304 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117307 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117309 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117312 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117315 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117331 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:17:42.167463 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117334 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117337 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117340 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117342 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117345 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117348 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117351 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117353 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117356 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117359 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117361 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117364 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117367 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117369 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117372 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117374 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117378 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117382 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117384 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:17:42.167907 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117387 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117390 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117393 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117395 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117398 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117400 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117402 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117405 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117407 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117413 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117415 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117419 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117423 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117426 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117429 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117431 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117434 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117437 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117440 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:17:42.229634 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117442 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117445 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117447 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117450 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117453 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117456 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117458 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117461 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117463 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117465 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117468 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117470 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117473 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117475 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117478 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117480 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117483 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117485 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117488 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117490 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:17:42.230355 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117493 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117495 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117499 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117501 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117504 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117506 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117509 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117511 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117514 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117517 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117519 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117523 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117526 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117530 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117532 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117535 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117538 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117542 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117545 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117547 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:17:42.231014 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117550 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117552 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117555 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117557 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117559 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117562 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117565 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117567 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117570 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117572 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117575 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117577 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.117583 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117689 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117695 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 17:17:42.231692 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117698 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117701 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117704 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117706 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117708 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117711 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117714 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117716 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117719 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117722 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117724 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117727 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117729 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117732 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117734 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117738 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117740 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117743 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117745 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117748 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 17:17:42.232135 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117750 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117753 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117755 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117759 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117763 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117766 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117769 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117772 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117774 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117777 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117779 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117783 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117785 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117788 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117790 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117792 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117795 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117797 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117801 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117804 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 17:17:42.232812 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117807 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117809 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117812 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117814 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117817 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117819 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117822 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117825 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117829 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117831 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117834 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117836 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117839 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117841 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117844 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117846 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117848 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117851 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117854 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117856 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 17:17:42.233486 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117859 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117862 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117864 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117868 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117871 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117874 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117876 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117879 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117881 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117884 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117886 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117888 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117891 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117900 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117904 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117906 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117909 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117911 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117913 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 17:17:42.234166 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117917 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117920 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117922 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117925 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117927 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117949 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117952 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117954 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117957 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117959 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117962 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117964 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117967 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117971 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117975 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117977 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117981 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 17:17:42.234847 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:42.117984 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.117989 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.118081 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.120411 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.121646 2579 server.go:1019] "Starting client certificate rotation" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.121916 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.121978 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.123983 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.130739 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.138484 2579 log.go:25] "Validated CRI v1 runtime API" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.140706 2579 log.go:25] "Validated CRI v1 image API" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.144475 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 17:17:42.235755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.148770 2579 fs.go:135] Filesystem UUIDs: map[5389195f-8c60-434f-9f7d-8b607d384121:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e4a8dae5-ad88-4e17-9b7c-7ed4206fe369:/dev/nvme0n1p3] Jul 13 17:17:42.236201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.148790 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 17:17:42.236640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.154991 2579 manager.go:216] Machine: {Timestamp:2026-07-13 17:17:42.153577428 +0000 UTC m=+0.310064300 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200357 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a403e478c09ac54f26854e1dba3da SystemUUID:ec2a403e-478c-09ac-54f2-6854e1dba3da BootID:fa7adf09-26c7-4dbc-b5e8-47a476cb88db Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:88:d0:fd:7a:bd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:88:d0:fd:7a:bd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4a:2b:ba:53:02:23 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 17:17:42.236640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.155170 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 17:17:42.236640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.155262 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 17:17:42.236640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.156302 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 17:17:42.236640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.156348 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-131.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 17:17:42.236640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.157230 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 17:17:42.236640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.157242 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 17:17:42.236640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.157252 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.157874 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.158830 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.162985 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.163126 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.163930 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.163952 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.163965 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.163975 2579 kubelet.go:398] "Adding apiserver pod source" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.163983 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.164463 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 17:17:42.237095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.164482 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.412312 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.413865 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.413901 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414562 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414580 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414596 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414602 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414612 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414617 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414625 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414631 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414638 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414644 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414656 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.414663 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.415311 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.415333 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.418588 2579 server.go:1268] "Started kubelet" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.418764 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.418838 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.418874 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.420062 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.420073 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.425228 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.425248 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.425387 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.425920 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.425926 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.425973 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.426920 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.426928 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.429660 2579 factory.go:55] Registering systemd factory Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.429676 2579 factory.go:223] Registration of the systemd container factory successfully Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.429883 2579 factory.go:153] Registering CRI-O factory Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.429904 2579 factory.go:223] Registration of the crio container factory successfully Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.430000 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.430024 2579 factory.go:103] Registering Raw factory Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.430040 2579 manager.go:1196] Started watching for new ooms in manager Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.430247 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.430395 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2mhwv" Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.430665 2579 manager.go:349] Starting recovery of all containers Jul 13 17:17:42.446616 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.430678 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 17:17:42.419633 ip-10-0-143-131 systemd[1]: Started Kubernetes Kubelet. Jul 13 17:17:42.448068 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.435125 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-131.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 17:17:42.448068 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.435267 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-131.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 17:17:42.448068 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.435501 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-131.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 17:17:42.448068 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.435667 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 17:17:42.448068 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.435846 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 17:17:42.448068 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.435885 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-131.ec2.internal.18c1e8f413e04bde default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-131.ec2.internal,UID:ip-10-0-143-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-131.ec2.internal,},FirstTimestamp:2026-07-13 17:17:42.418557918 +0000 UTC m=+0.575044784,LastTimestamp:2026-07-13 17:17:42.418557918 +0000 UTC m=+0.575044784,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-131.ec2.internal,}" Jul 13 17:17:42.448068 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.439802 2579 manager.go:354] Recovery completed Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.451786 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.455358 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.455383 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.455396 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.460391 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.460401 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.460403 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.460418 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.461122 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.461143 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.461152 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.462477 2579 policy_none.go:49] "None policy: Start" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.462489 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.462499 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.463257 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-131.ec2.internal.18c1e8f41611ff76 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-131.ec2.internal,UID:ip-10-0-143-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-131.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-131.ec2.internal,},FirstTimestamp:2026-07-13 17:17:42.45536959 +0000 UTC m=+0.611856458,LastTimestamp:2026-07-13 17:17:42.45536959 +0000 UTC m=+0.611856458,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-131.ec2.internal,}" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.464427 2579 policy_none.go:47] "Start" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.515942 2579 manager.go:337] "Starting Device Plugin manager" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.515969 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.515978 2579 server.go:87] "Starting device plugin registration server" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.516193 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.516206 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.516280 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.516349 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.516357 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.516773 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 17:17:42.585005 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.516823 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.617709 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.619396 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.619426 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.619437 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.619457 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.628762 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.628778 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-131.ec2.internal\": node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.653250 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.654909 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.654944 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.654965 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.655001 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.658584 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.677311 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.756124 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal"] Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.756189 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.758532 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.758982 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.759003 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.761608 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.761712 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.761740 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.762367 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.762388 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.762398 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.762461 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.762482 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:17:42.763769 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.762497 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:17:42.765385 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.764967 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.765385 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.764995 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:17:42.767490 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.765743 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:17:42.767490 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.765770 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:17:42.767490 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.765788 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.778693 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.797137 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-131.ec2.internal\" not found" node="ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.801539 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-131.ec2.internal\" not found" node="ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.828965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6d457d90ba56949705c8ce1904d5667-config\") pod \"kube-apiserver-proxy-ip-10-0-143-131.ec2.internal\" (UID: \"e6d457d90ba56949705c8ce1904d5667\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.828992 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31b6393229f8bfa309050cf3ed4b301e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal\" (UID: \"31b6393229f8bfa309050cf3ed4b301e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.829011 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31b6393229f8bfa309050cf3ed4b301e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal\" (UID: \"31b6393229f8bfa309050cf3ed4b301e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.880667 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.929589 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6d457d90ba56949705c8ce1904d5667-config\") pod \"kube-apiserver-proxy-ip-10-0-143-131.ec2.internal\" (UID: \"e6d457d90ba56949705c8ce1904d5667\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.929614 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31b6393229f8bfa309050cf3ed4b301e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal\" (UID: \"31b6393229f8bfa309050cf3ed4b301e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.929632 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31b6393229f8bfa309050cf3ed4b301e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal\" (UID: \"31b6393229f8bfa309050cf3ed4b301e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.929674 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31b6393229f8bfa309050cf3ed4b301e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal\" (UID: \"31b6393229f8bfa309050cf3ed4b301e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.929692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e6d457d90ba56949705c8ce1904d5667-config\") pod \"kube-apiserver-proxy-ip-10-0-143-131.ec2.internal\" (UID: \"e6d457d90ba56949705c8ce1904d5667\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" Jul 13 17:17:42.936371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:42.929697 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31b6393229f8bfa309050cf3ed4b301e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal\" (UID: \"31b6393229f8bfa309050cf3ed4b301e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:43.117744 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:42.980748 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:43.117744 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:43.083233 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:43.117744 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.100317 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:43.117744 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.110216 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" Jul 13 17:17:43.121713 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.121501 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 17:17:43.121713 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.121657 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 17:17:43.121713 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:43.121609 2579 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a880f5a4aff3b48e09830f94f9eb529f-3a789dda0551d3a1.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-143-131.ec2.internal.18c1e8f41611ff76\": read tcp 10.0.143.131:43826->54.211.194.238:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-143-131.ec2.internal.18c1e8f41611ff76 default 6369 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-131.ec2.internal,UID:ip-10-0-143-131.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-131.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-131.ec2.internal,},FirstTimestamp:2026-07-13 17:17:42 +0000 UTC,LastTimestamp:2026-07-13 17:17:42.765755162 +0000 UTC m=+0.922242031,Count:7,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-131.ec2.internal,}" Jul 13 17:17:43.315200 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:43.184250 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:43.315200 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:43.288568 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:43.447925 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:43.388922 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:43.447925 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.439611 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 17:12:42 +0000 UTC" deadline="2027-12-11 03:17:35.555514822 +0000 UTC" Jul 13 17:17:43.447925 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.439645 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12369h59m52.115873293s" Jul 13 17:17:43.623468 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:43.489634 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:43.623468 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.546509 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 17:17:43.623468 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.547352 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientMemory" Jul 13 17:17:43.623468 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.547374 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 17:17:43.623468 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.547384 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeHasSufficientPID" Jul 13 17:17:43.623468 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.553630 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:17:43.623468 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:43.590203 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-131.ec2.internal\" not found" Jul 13 17:17:43.623468 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.597461 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:17:43.626544 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.626238 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" Jul 13 17:17:43.804012 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.656924 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 17:17:43.804012 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.658715 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" Jul 13 17:17:43.804012 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.704886 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 17:17:43.804012 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:43.751899 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d457d90ba56949705c8ce1904d5667.slice/crio-7add1c6d72c2cf2fd8c35264dd00577aa6a86fb59e4982258bc18f4ca09400f7 WatchSource:0}: Error finding container 7add1c6d72c2cf2fd8c35264dd00577aa6a86fb59e4982258bc18f4ca09400f7: Status 404 returned error can't find the container with id 7add1c6d72c2cf2fd8c35264dd00577aa6a86fb59e4982258bc18f4ca09400f7 Jul 13 17:17:43.804012 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.761484 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:17:43.804012 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:43.766559 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b6393229f8bfa309050cf3ed4b301e.slice/crio-4d4137286543375bfce9d2ce52031a44bb856ad925ad7e58bebcd525bb5faff8 WatchSource:0}: Error finding container 4d4137286543375bfce9d2ce52031a44bb856ad925ad7e58bebcd525bb5faff8: Status 404 returned error can't find the container with id 4d4137286543375bfce9d2ce52031a44bb856ad925ad7e58bebcd525bb5faff8 Jul 13 17:17:43.985916 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:43.836014 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.167070 2579 apiserver.go:52] "Watching apiserver" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.176231 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.176581 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-h9tmj","openshift-dns/node-resolver-kl9vm","openshift-multus/multus-additional-cni-plugins-cwbb8","openshift-multus/multus-j2dbs","openshift-image-registry/node-ca-bqkvc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal","openshift-multus/network-metrics-daemon-48dkb","openshift-network-diagnostics/network-check-target-f2p4s","openshift-network-operator/iptables-alerter-xch94","openshift-ovn-kubernetes/ovnkube-node-snh5v","kube-system/global-pull-secret-syncer-twrr7","kube-system/konnectivity-agent-z96wx","kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw"] Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.179380 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.181601 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.182796 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.183530 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.183549 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.183647 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.183662 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.183660 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vnd2c\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.183917 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.183947 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.184109 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7krws\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.186159 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.186236 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.188505 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.189372 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rk8tm\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.190178 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.190912 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.191992 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.192185 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.192357 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.193928 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:44.196687 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.193975 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:44.198972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.197138 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.198972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.197596 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 17:17:44.198972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.197619 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 17:17:44.198972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.197669 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 17:17:44.198972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.197680 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-gt2zw\"" Jul 13 17:17:44.198972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.197951 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 17:17:44.198972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.198155 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-72vfs\"" Jul 13 17:17:44.198972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.198210 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.200750 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.200854 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.203984 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.206533 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.206871 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.207011 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.207150 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gsbzk\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.207664 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.207893 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dzjjg\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.207935 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.208170 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.208405 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.208754 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.208891 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b74ph\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.208967 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9mg2h\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.209123 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.209169 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.211147 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.211699 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.226740 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.228423 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.228447 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238209 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-lib-modules\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238232 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-var-lib-kubelet\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.363148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac6adcdd-a342-4652-8762-af45ca4d1114-ovn-node-metrics-cert\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238268 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-conf-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-tuned\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238312 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13d2222c-9fec-4ae0-849a-49f4bf8b21e3-agent-certs\") pod \"konnectivity-agent-z96wx\" (UID: \"13d2222c-9fec-4ae0-849a-49f4bf8b21e3\") " pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-cni-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48zhz\" (UniqueName: \"kubernetes.io/projected/e411827e-84c9-4197-b166-8e0868a0709d-kube-api-access-48zhz\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238408 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-socket-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238435 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35d03ed5-a235-40aa-815f-b8f286ba77c3-host\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238450 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-ovnkube-config\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238465 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nnpx\" (UniqueName: \"kubernetes.io/projected/cf7543c5-daf6-4ad6-9d05-d00958baac07-kube-api-access-4nnpx\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238486 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn4rx\" (UniqueName: \"kubernetes.io/projected/35d03ed5-a235-40aa-815f-b8f286ba77c3-kube-api-access-zn4rx\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238519 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cni-binary-copy\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238552 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-slash\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.364555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238577 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-ovn\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238606 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-dbus\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238650 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysconfig\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-systemd\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-os-release\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238723 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e411827e-84c9-4197-b166-8e0868a0709d-cni-binary-copy\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238751 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-modprobe-d\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238773 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238805 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-systemd-units\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238840 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-cni-netd\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpgwp\" (UniqueName: \"kubernetes.io/projected/ac6adcdd-a342-4652-8762-af45ca4d1114-kube-api-access-gpgwp\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238903 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkj9v\" (UniqueName: \"kubernetes.io/projected/87c1ac36-6870-4e28-97a9-1df7abf70a5e-kube-api-access-kkj9v\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-etc-kubernetes\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238943 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysctl-conf\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.456322 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.238980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-run\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239027 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239063 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-run-netns\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239121 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-ovnkube-script-lib\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-k8s-cni-cncf-io\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239173 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-kubelet\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239200 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-kubernetes\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239226 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-sys\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239250 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239277 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-var-lib-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239296 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-cni-bin\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239312 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-hostroot\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.457683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239327 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-registration-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239383 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239415 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-host\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239442 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-host-slash\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239471 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysctl-d\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239499 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-etc-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239525 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239561 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-multus-certs\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239590 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf7543c5-daf6-4ad6-9d05-d00958baac07-tmp-dir\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239616 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-device-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239651 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-595vv\" (UniqueName: \"kubernetes.io/projected/65942462-cf09-4171-8e03-8fa923e57a3b-kube-api-access-595vv\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239679 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35d03ed5-a235-40aa-815f-b8f286ba77c3-serviceca\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmdtg\" (UniqueName: \"kubernetes.io/projected/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-kube-api-access-rmdtg\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.459086 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58nzk\" (UniqueName: \"kubernetes.io/projected/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-kube-api-access-58nzk\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239801 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-iptables-alerter-script\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239826 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-system-cni-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239850 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-socket-dir-parent\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239881 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-netns\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239968 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-os-release\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.239999 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240021 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-cni-multus\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240045 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e411827e-84c9-4197-b166-8e0868a0709d-multus-daemon-config\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240067 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-systemd\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240089 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-tmp\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240166 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cnibin\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240191 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-node-log\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240216 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.460725 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13d2222c-9fec-4ae0-849a-49f4bf8b21e3-konnectivity-ca\") pod \"konnectivity-agent-z96wx\" (UID: \"13d2222c-9fec-4ae0-849a-49f4bf8b21e3\") " pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240264 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-cnibin\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-cni-bin\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240309 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-system-cni-dir\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240337 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbjwx\" (UniqueName: \"kubernetes.io/projected/bd72547b-0a43-40f8-8ec6-ea60f1634e41-kube-api-access-sbjwx\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240363 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-kubelet\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240494 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-log-socket\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240525 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-run-ovn-kubernetes\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240565 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf7543c5-daf6-4ad6-9d05-d00958baac07-hosts-file\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240617 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-kubelet-config\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240666 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-sys-fs\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-env-overrides\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.240752 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-systemd\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.461668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341650 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-tmp\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341670 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cnibin\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-node-log\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341714 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341743 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-systemd\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341766 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-node-log\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341802 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13d2222c-9fec-4ae0-849a-49f4bf8b21e3-konnectivity-ca\") pod \"konnectivity-agent-z96wx\" (UID: \"13d2222c-9fec-4ae0-849a-49f4bf8b21e3\") " pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341868 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cnibin\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341954 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-cnibin\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.341994 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-cni-bin\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-system-cni-dir\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342042 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbjwx\" (UniqueName: \"kubernetes.io/projected/bd72547b-0a43-40f8-8ec6-ea60f1634e41-kube-api-access-sbjwx\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342065 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-kubelet\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.463046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342067 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-cnibin\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342090 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-log-socket\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342103 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342115 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-system-cni-dir\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342135 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-log-socket\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-run-ovn-kubernetes\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342168 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf7543c5-daf6-4ad6-9d05-d00958baac07-hosts-file\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342173 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-kubelet\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342194 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-kubelet-config\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342219 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-sys-fs\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342219 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-run-ovn-kubernetes\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342245 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-env-overrides\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342287 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-kubelet-config\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342292 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342330 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-lib-modules\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.464002 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342376 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-sys-fs\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342381 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-var-lib-kubelet\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.342425 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342429 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac6adcdd-a342-4652-8762-af45ca4d1114-ovn-node-metrics-cert\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342348 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf7543c5-daf6-4ad6-9d05-d00958baac07-hosts-file\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342495 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-lib-modules\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342455 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-conf-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342496 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13d2222c-9fec-4ae0-849a-49f4bf8b21e3-konnectivity-ca\") pod \"konnectivity-agent-z96wx\" (UID: \"13d2222c-9fec-4ae0-849a-49f4bf8b21e3\") " pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342522 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-conf-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.342537 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret podName:50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9 nodeName:}" failed. No retries permitted until 2026-07-13 17:17:44.842514985 +0000 UTC m=+2.999001854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret") pod "global-pull-secret-syncer-twrr7" (UID: "50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342559 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-tuned\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342568 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-var-lib-kubelet\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13d2222c-9fec-4ae0-849a-49f4bf8b21e3-agent-certs\") pod \"konnectivity-agent-z96wx\" (UID: \"13d2222c-9fec-4ae0-849a-49f4bf8b21e3\") " pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342112 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-cni-bin\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342642 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-cni-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.465137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342705 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-cni-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342733 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48zhz\" (UniqueName: \"kubernetes.io/projected/e411827e-84c9-4197-b166-8e0868a0709d-kube-api-access-48zhz\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342756 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342777 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-socket-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35d03ed5-a235-40aa-815f-b8f286ba77c3-host\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342823 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-ovnkube-config\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342833 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nnpx\" (UniqueName: \"kubernetes.io/projected/cf7543c5-daf6-4ad6-9d05-d00958baac07-kube-api-access-4nnpx\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342874 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn4rx\" (UniqueName: \"kubernetes.io/projected/35d03ed5-a235-40aa-815f-b8f286ba77c3-kube-api-access-zn4rx\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342896 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cni-binary-copy\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-slash\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342943 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-ovn\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342964 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-dbus\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.342988 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysconfig\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.466963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-systemd\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343030 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-os-release\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e411827e-84c9-4197-b166-8e0868a0709d-cni-binary-copy\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-modprobe-d\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343120 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-systemd-units\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343142 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-cni-netd\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343168 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpgwp\" (UniqueName: \"kubernetes.io/projected/ac6adcdd-a342-4652-8762-af45ca4d1114-kube-api-access-gpgwp\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343192 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkj9v\" (UniqueName: \"kubernetes.io/projected/87c1ac36-6870-4e28-97a9-1df7abf70a5e-kube-api-access-kkj9v\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343214 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-etc-kubernetes\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysctl-conf\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-run\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343259 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-socket-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.467644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343307 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35d03ed5-a235-40aa-815f-b8f286ba77c3-host\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343308 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343344 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-run-netns\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343388 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-ovnkube-script-lib\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343415 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-k8s-cni-cncf-io\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-kubelet\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343472 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-kubernetes\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343497 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-sys\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343521 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343547 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-var-lib-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343570 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-cni-bin\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-hostroot\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343619 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-registration-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343644 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.468791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-host\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-host-slash\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343716 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysctl-d\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343736 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-etc-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343758 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-multus-certs\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343803 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf7543c5-daf6-4ad6-9d05-d00958baac07-tmp-dir\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.343804 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344157 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-cni-bin\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344222 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cni-binary-copy\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344263 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-device-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344291 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-595vv\" (UniqueName: \"kubernetes.io/projected/65942462-cf09-4171-8e03-8fa923e57a3b-kube-api-access-595vv\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344316 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35d03ed5-a235-40aa-815f-b8f286ba77c3-serviceca\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmdtg\" (UniqueName: \"kubernetes.io/projected/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-kube-api-access-rmdtg\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.471151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58nzk\" (UniqueName: \"kubernetes.io/projected/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-kube-api-access-58nzk\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344391 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-iptables-alerter-script\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344431 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-system-cni-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344448 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-run-netns\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-socket-dir-parent\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344486 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-netns\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-os-release\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344533 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-cni-multus\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.344581 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e411827e-84c9-4197-b166-8e0868a0709d-multus-daemon-config\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345077 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e411827e-84c9-4197-b166-8e0868a0709d-multus-daemon-config\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345104 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-kubernetes\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345131 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-slash\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345154 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-k8s-cni-cncf-io\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345176 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-ovn\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.471789 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345194 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-kubelet\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345236 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-systemd\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345289 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-dbus\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345293 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-os-release\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345336 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysconfig\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345402 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-device-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345486 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-modprobe-d\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345968 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bd72547b-0a43-40f8-8ec6-ea60f1634e41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.345977 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-sys\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346077 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-host-cni-netd\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346210 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-var-lib-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346257 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-systemd-units\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346432 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e411827e-84c9-4197-b166-8e0868a0709d-cni-binary-copy\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346450 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-tmp\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346459 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13d2222c-9fec-4ae0-849a-49f4bf8b21e3-agent-certs\") pod \"konnectivity-agent-z96wx\" (UID: \"13d2222c-9fec-4ae0-849a-49f4bf8b21e3\") " pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:17:44.472479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346521 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-etc-kubernetes\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346528 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-system-cni-dir\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346544 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-os-release\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346594 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-netns\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346631 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-run-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346643 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysctl-conf\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346668 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-var-lib-cni-multus\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346706 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-run\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346868 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd72547b-0a43-40f8-8ec6-ea60f1634e41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346893 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35d03ed5-a235-40aa-815f-b8f286ba77c3-serviceca\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346934 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-host-slash\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.346984 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-multus-socket-dir-parent\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347004 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-hostroot\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.347005 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347043 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac6adcdd-a342-4652-8762-af45ca4d1114-etc-openvswitch\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.473238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347079 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-sysctl-d\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.347083 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:17:44.847061713 +0000 UTC m=+3.003548573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347126 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-etc-selinux\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347131 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-host\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347134 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e411827e-84c9-4197-b166-8e0868a0709d-host-run-multus-certs\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347176 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/65942462-cf09-4171-8e03-8fa923e57a3b-registration-dir\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347179 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-iptables-alerter-script\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347297 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-etc-tuned\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.347383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf7543c5-daf6-4ad6-9d05-d00958baac07-tmp-dir\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.353747 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-env-overrides\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.353751 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac6adcdd-a342-4652-8762-af45ca4d1114-ovn-node-metrics-cert\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.353853 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-ovnkube-config\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.353891 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac6adcdd-a342-4652-8762-af45ca4d1114-ovnkube-script-lib\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.369266 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkj9v\" (UniqueName: \"kubernetes.io/projected/87c1ac36-6870-4e28-97a9-1df7abf70a5e-kube-api-access-kkj9v\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.375989 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:17:44.474198 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.376007 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.376017 2579 projected.go:196] Error preparing data for projected volume kube-api-access-98gzk for pod openshift-network-diagnostics/network-check-target-f2p4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.376086 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk podName:721418c3-65df-44e7-8c2a-3f2395cfff5a nodeName:}" failed. No retries permitted until 2026-07-13 17:17:44.876072672 +0000 UTC m=+3.032559526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-98gzk" (UniqueName: "kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk") pod "network-check-target-f2p4s" (UID: "721418c3-65df-44e7-8c2a-3f2395cfff5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.393073 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58nzk\" (UniqueName: \"kubernetes.io/projected/d3dc6950-ef2c-4fa2-b392-9483a8a3faed-kube-api-access-58nzk\") pod \"tuned-h9tmj\" (UID: \"d3dc6950-ef2c-4fa2-b392-9483a8a3faed\") " pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.393287 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn4rx\" (UniqueName: \"kubernetes.io/projected/35d03ed5-a235-40aa-815f-b8f286ba77c3-kube-api-access-zn4rx\") pod \"node-ca-bqkvc\" (UID: \"35d03ed5-a235-40aa-815f-b8f286ba77c3\") " pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.393307 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-595vv\" (UniqueName: \"kubernetes.io/projected/65942462-cf09-4171-8e03-8fa923e57a3b-kube-api-access-595vv\") pod \"aws-ebs-csi-driver-node-6qlrw\" (UID: \"65942462-cf09-4171-8e03-8fa923e57a3b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.393399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpgwp\" (UniqueName: \"kubernetes.io/projected/ac6adcdd-a342-4652-8762-af45ca4d1114-kube-api-access-gpgwp\") pod \"ovnkube-node-snh5v\" (UID: \"ac6adcdd-a342-4652-8762-af45ca4d1114\") " pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.394539 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nnpx\" (UniqueName: \"kubernetes.io/projected/cf7543c5-daf6-4ad6-9d05-d00958baac07-kube-api-access-4nnpx\") pod \"node-resolver-kl9vm\" (UID: \"cf7543c5-daf6-4ad6-9d05-d00958baac07\") " pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.395856 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbjwx\" (UniqueName: \"kubernetes.io/projected/bd72547b-0a43-40f8-8ec6-ea60f1634e41-kube-api-access-sbjwx\") pod \"multus-additional-cni-plugins-cwbb8\" (UID: \"bd72547b-0a43-40f8-8ec6-ea60f1634e41\") " pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.397479 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48zhz\" (UniqueName: \"kubernetes.io/projected/e411827e-84c9-4197-b166-8e0868a0709d-kube-api-access-48zhz\") pod \"multus-j2dbs\" (UID: \"e411827e-84c9-4197-b166-8e0868a0709d\") " pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.398031 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmdtg\" (UniqueName: \"kubernetes.io/projected/3456e2f2-809f-4ccf-8b3d-3c0ddf53c583-kube-api-access-rmdtg\") pod \"iptables-alerter-xch94\" (UID: \"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583\") " pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.426508 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 17:17:44.474906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.453871 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 17:17:44.489085 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.489058 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bqkvc" Jul 13 17:17:44.498570 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.498288 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xch94" Jul 13 17:17:44.500336 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:44.500283 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d03ed5_a235_40aa_815f_b8f286ba77c3.slice/crio-a45051c859d754fffaad1190c78be17e3777143e7002f221fa143c350f23be23 WatchSource:0}: Error finding container a45051c859d754fffaad1190c78be17e3777143e7002f221fa143c350f23be23: Status 404 returned error can't find the container with id a45051c859d754fffaad1190c78be17e3777143e7002f221fa143c350f23be23 Jul 13 17:17:44.507180 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.507023 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" Jul 13 17:17:44.513143 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.513121 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j2dbs" Jul 13 17:17:44.521038 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:44.521013 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3456e2f2_809f_4ccf_8b3d_3c0ddf53c583.slice/crio-d188207c701d5c46a5bbad76ed54487af64784e76994f32da0ff34cfb61a1873 WatchSource:0}: Error finding container d188207c701d5c46a5bbad76ed54487af64784e76994f32da0ff34cfb61a1873: Status 404 returned error can't find the container with id d188207c701d5c46a5bbad76ed54487af64784e76994f32da0ff34cfb61a1873 Jul 13 17:17:44.522152 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.521957 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" Jul 13 17:17:44.527010 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.526989 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kl9vm" Jul 13 17:17:44.532745 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.532725 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:17:44.538657 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.538633 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:17:44.539531 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:44.539505 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd72547b_0a43_40f8_8ec6_ea60f1634e41.slice/crio-0d908be4ae1326f7c7aa00eca12bce4fa5afa068097be06e3599e94f7b35ae29 WatchSource:0}: Error finding container 0d908be4ae1326f7c7aa00eca12bce4fa5afa068097be06e3599e94f7b35ae29: Status 404 returned error can't find the container with id 0d908be4ae1326f7c7aa00eca12bce4fa5afa068097be06e3599e94f7b35ae29 Jul 13 17:17:44.544356 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:44.544322 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7543c5_daf6_4ad6_9d05_d00958baac07.slice/crio-3d35e586c3841f645aff2cf02b87fb6d9c13d0c40be59d41aa2942ead19885a8 WatchSource:0}: Error finding container 3d35e586c3841f645aff2cf02b87fb6d9c13d0c40be59d41aa2942ead19885a8: Status 404 returned error can't find the container with id 3d35e586c3841f645aff2cf02b87fb6d9c13d0c40be59d41aa2942ead19885a8 Jul 13 17:17:44.545453 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.545436 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" Jul 13 17:17:44.557530 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:44.557507 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6adcdd_a342_4652_8762_af45ca4d1114.slice/crio-a5ece175dccab5c052fe5c6d58fe2242fbdc90c7716c89b2f9e7d86901cda6b1 WatchSource:0}: Error finding container a5ece175dccab5c052fe5c6d58fe2242fbdc90c7716c89b2f9e7d86901cda6b1: Status 404 returned error can't find the container with id a5ece175dccab5c052fe5c6d58fe2242fbdc90c7716c89b2f9e7d86901cda6b1 Jul 13 17:17:44.567430 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:17:44.567405 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65942462_cf09_4171_8e03_8fa923e57a3b.slice/crio-ce9ea2e282f3fe7827c1159f459e67fb9f52ac9f8ab6ba65d3656687fc293bf9 WatchSource:0}: Error finding container ce9ea2e282f3fe7827c1159f459e67fb9f52ac9f8ab6ba65d3656687fc293bf9: Status 404 returned error can't find the container with id ce9ea2e282f3fe7827c1159f459e67fb9f52ac9f8ab6ba65d3656687fc293bf9 Jul 13 17:17:44.570216 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.570078 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mknbt" Jul 13 17:17:44.591932 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.591580 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mknbt" Jul 13 17:17:44.661301 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.661200 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" event={"ID":"65942462-cf09-4171-8e03-8fa923e57a3b","Type":"ContainerStarted","Data":"ce9ea2e282f3fe7827c1159f459e67fb9f52ac9f8ab6ba65d3656687fc293bf9"} Jul 13 17:17:44.664150 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.664124 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z96wx" event={"ID":"13d2222c-9fec-4ae0-849a-49f4bf8b21e3","Type":"ContainerStarted","Data":"84d27965513d09a7513ec6c393f596f8dcfd417ad1bc37d24f2c36557dbe8e7d"} Jul 13 17:17:44.665728 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.665647 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"a5ece175dccab5c052fe5c6d58fe2242fbdc90c7716c89b2f9e7d86901cda6b1"} Jul 13 17:17:44.667030 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.667006 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kl9vm" event={"ID":"cf7543c5-daf6-4ad6-9d05-d00958baac07","Type":"ContainerStarted","Data":"3d35e586c3841f645aff2cf02b87fb6d9c13d0c40be59d41aa2942ead19885a8"} Jul 13 17:17:44.668397 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.668318 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerStarted","Data":"0d908be4ae1326f7c7aa00eca12bce4fa5afa068097be06e3599e94f7b35ae29"} Jul 13 17:17:44.669602 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.669578 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xch94" event={"ID":"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583","Type":"ContainerStarted","Data":"d188207c701d5c46a5bbad76ed54487af64784e76994f32da0ff34cfb61a1873"} Jul 13 17:17:44.670196 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.670176 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 17:17:44.671219 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.671196 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" event={"ID":"31b6393229f8bfa309050cf3ed4b301e","Type":"ContainerStarted","Data":"4d4137286543375bfce9d2ce52031a44bb856ad925ad7e58bebcd525bb5faff8"} Jul 13 17:17:44.672823 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.672584 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" event={"ID":"e6d457d90ba56949705c8ce1904d5667","Type":"ContainerStarted","Data":"7add1c6d72c2cf2fd8c35264dd00577aa6a86fb59e4982258bc18f4ca09400f7"} Jul 13 17:17:44.674388 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.674364 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j2dbs" event={"ID":"e411827e-84c9-4197-b166-8e0868a0709d","Type":"ContainerStarted","Data":"51fe416ede14eb33ae0342f009a173287ec480e903ff9d7beffa2aaf2d29f7cd"} Jul 13 17:17:44.675827 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.675804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" event={"ID":"d3dc6950-ef2c-4fa2-b392-9483a8a3faed","Type":"ContainerStarted","Data":"6d03b8e22e390d9e0e4ed19752dc5648caed940be9388724523c655aa9c0bd18"} Jul 13 17:17:44.677148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.677125 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bqkvc" event={"ID":"35d03ed5-a235-40aa-815f-b8f286ba77c3","Type":"ContainerStarted","Data":"a45051c859d754fffaad1190c78be17e3777143e7002f221fa143c350f23be23"} Jul 13 17:17:44.849821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.849730 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:44.849821 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.849797 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:44.850048 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.849903 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:44.850048 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.849906 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:44.850048 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.849966 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret podName:50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9 nodeName:}" failed. No retries permitted until 2026-07-13 17:17:45.849948346 +0000 UTC m=+4.006435207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret") pod "global-pull-secret-syncer-twrr7" (UID: "50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:44.850048 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.849986 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:17:45.84997536 +0000 UTC m=+4.006462221 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:44.950782 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:44.950741 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:44.950947 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.950926 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:17:44.951009 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.950951 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:17:44.951009 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.950966 2579 projected.go:196] Error preparing data for projected volume kube-api-access-98gzk for pod openshift-network-diagnostics/network-check-target-f2p4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:44.951119 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:44.951029 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk podName:721418c3-65df-44e7-8c2a-3f2395cfff5a nodeName:}" failed. No retries permitted until 2026-07-13 17:17:45.951009911 +0000 UTC m=+4.107496786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-98gzk" (UniqueName: "kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk") pod "network-check-target-f2p4s" (UID: "721418c3-65df-44e7-8c2a-3f2395cfff5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:45.596515 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:45.596469 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 17:12:44 +0000 UTC" deadline="2028-03-23 19:10:11.2276558 +0000 UTC" Jul 13 17:17:45.596515 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:45.596514 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14857h52m25.63114712s" Jul 13 17:17:45.655377 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:45.655347 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:45.655553 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.655471 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:45.655553 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:45.655512 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:45.655729 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.655575 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:45.655729 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:45.655658 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:45.655854 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.655761 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:45.858806 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:45.858324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:45.858806 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:45.858408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:45.858806 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.858543 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:45.858806 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.858617 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret podName:50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9 nodeName:}" failed. No retries permitted until 2026-07-13 17:17:47.858596892 +0000 UTC m=+6.015083751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret") pod "global-pull-secret-syncer-twrr7" (UID: "50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:45.858806 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.858691 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:45.858806 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.858725 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:17:47.858713454 +0000 UTC m=+6.015200314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:45.959472 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:45.959118 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:45.959472 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.959388 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:17:45.959472 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.959409 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:17:45.959472 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.959421 2579 projected.go:196] Error preparing data for projected volume kube-api-access-98gzk for pod openshift-network-diagnostics/network-check-target-f2p4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:45.959472 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:45.959474 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk podName:721418c3-65df-44e7-8c2a-3f2395cfff5a nodeName:}" failed. No retries permitted until 2026-07-13 17:17:47.959458175 +0000 UTC m=+6.115945043 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-98gzk" (UniqueName: "kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk") pod "network-check-target-f2p4s" (UID: "721418c3-65df-44e7-8c2a-3f2395cfff5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:46.597119 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:46.597068 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 17:12:44 +0000 UTC" deadline="2027-12-12 11:48:29.958089419 +0000 UTC" Jul 13 17:17:46.597119 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:46.597106 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12402h30m43.360989071s" Jul 13 17:17:47.655677 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:47.655639 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:47.656138 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.655781 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:47.656138 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:47.655858 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:47.656138 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.655936 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:47.656138 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:47.655985 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:47.656138 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.656048 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:47.873914 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:47.873880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:47.874077 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:47.873969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:47.874133 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.874087 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:47.874196 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.874140 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:17:51.874122169 +0000 UTC m=+10.030609027 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:47.874266 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.874229 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:47.874266 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.874260 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret podName:50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9 nodeName:}" failed. No retries permitted until 2026-07-13 17:17:51.874248626 +0000 UTC m=+10.030735478 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret") pod "global-pull-secret-syncer-twrr7" (UID: "50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:47.974858 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:47.974773 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:47.975018 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.974980 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:17:47.975018 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.975003 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:17:47.975018 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.975016 2579 projected.go:196] Error preparing data for projected volume kube-api-access-98gzk for pod openshift-network-diagnostics/network-check-target-f2p4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:47.975176 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:47.975071 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk podName:721418c3-65df-44e7-8c2a-3f2395cfff5a nodeName:}" failed. No retries permitted until 2026-07-13 17:17:51.975053976 +0000 UTC m=+10.131540850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-98gzk" (UniqueName: "kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk") pod "network-check-target-f2p4s" (UID: "721418c3-65df-44e7-8c2a-3f2395cfff5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:49.655797 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:49.655567 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:49.655797 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:49.655765 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:49.655797 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:49.655567 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:49.656283 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:49.655877 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:49.656283 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:49.655567 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:49.656283 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:49.655972 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:50.697411 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:50.697306 2579 generic.go:356] "Generic (PLEG): container finished" podID="31b6393229f8bfa309050cf3ed4b301e" containerID="e5c95fc4b4974a307beb7c5d1d0e9aba40a1e60c7382726675bfa07f4cefd065" exitCode=0 Jul 13 17:17:50.697411 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:50.697409 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" event={"ID":"31b6393229f8bfa309050cf3ed4b301e","Type":"ContainerDied","Data":"e5c95fc4b4974a307beb7c5d1d0e9aba40a1e60c7382726675bfa07f4cefd065"} Jul 13 17:17:51.656208 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:51.656173 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:51.656425 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:51.656304 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:51.656425 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:51.656333 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:51.656425 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:51.656312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:51.656602 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:51.656447 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:51.656602 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:51.656506 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:51.907842 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:51.907646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:51.907842 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:51.907726 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:51.907842 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:51.907797 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:51.907842 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:51.907842 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:51.908427 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:51.907871 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:17:59.907853292 +0000 UTC m=+18.064340167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:51.908427 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:51.907891 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret podName:50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9 nodeName:}" failed. No retries permitted until 2026-07-13 17:17:59.907881356 +0000 UTC m=+18.064368208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret") pod "global-pull-secret-syncer-twrr7" (UID: "50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:52.008332 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:52.008194 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:52.008492 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:52.008370 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:17:52.008492 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:52.008399 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:17:52.008492 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:52.008414 2579 projected.go:196] Error preparing data for projected volume kube-api-access-98gzk for pod openshift-network-diagnostics/network-check-target-f2p4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:52.008492 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:52.008477 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk podName:721418c3-65df-44e7-8c2a-3f2395cfff5a nodeName:}" failed. No retries permitted until 2026-07-13 17:18:00.00845621 +0000 UTC m=+18.164943067 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-98gzk" (UniqueName: "kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk") pod "network-check-target-f2p4s" (UID: "721418c3-65df-44e7-8c2a-3f2395cfff5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:17:53.656214 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:53.656178 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:53.656660 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:53.656425 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:53.656660 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:53.656302 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:53.656660 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:53.656509 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:53.656660 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:53.656276 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:53.656660 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:53.656596 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:55.655555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:55.655522 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:55.655555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:55.655555 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:55.656068 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:55.655557 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:55.656068 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:55.655652 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:55.656068 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:55.655724 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:55.656068 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:55.655801 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:57.655455 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:57.655417 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:57.655871 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:57.655520 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:57.655871 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:57.655531 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:57.655871 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:57.655547 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:57.655871 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:57.655630 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:57.655871 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:57.655707 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:59.655474 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:59.655440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:59.655474 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:59.655457 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:59.655917 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:59.655580 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:17:59.655917 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:59.655584 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:17:59.655917 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:59.655690 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:17:59.655917 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:59.655778 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:17:59.968398 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:59.968301 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:17:59.968398 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:17:59.968393 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:17:59.968626 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:59.968439 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:59.968626 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:59.968504 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:17:59.968626 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:59.968510 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret podName:50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9 nodeName:}" failed. No retries permitted until 2026-07-13 17:18:15.968488511 +0000 UTC m=+34.124975365 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret") pod "global-pull-secret-syncer-twrr7" (UID: "50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:17:59.968626 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:17:59.968568 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:18:15.968551246 +0000 UTC m=+34.125038105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:18:00.068689 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:00.068659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:00.068864 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:00.068843 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:18:00.068924 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:00.068874 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:18:00.068924 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:00.068889 2579 projected.go:196] Error preparing data for projected volume kube-api-access-98gzk for pod openshift-network-diagnostics/network-check-target-f2p4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:18:00.069010 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:00.068942 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk podName:721418c3-65df-44e7-8c2a-3f2395cfff5a nodeName:}" failed. No retries permitted until 2026-07-13 17:18:16.068927235 +0000 UTC m=+34.225414093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-98gzk" (UniqueName: "kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk") pod "network-check-target-f2p4s" (UID: "721418c3-65df-44e7-8c2a-3f2395cfff5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:18:01.655684 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:01.655652 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:01.655684 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:01.655682 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:01.656143 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:01.655775 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:01.656143 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:01.655803 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:01.656143 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:01.655889 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:01.656143 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:01.655967 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:03.655539 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.655440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:03.655879 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:03.655551 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:03.655879 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.655610 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:03.655879 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:03.655672 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:03.655879 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.655722 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:03.655879 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:03.655830 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:03.723088 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.722848 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" event={"ID":"31b6393229f8bfa309050cf3ed4b301e","Type":"ContainerStarted","Data":"98448205d046259936e2a85d95698b2a1bebaa146f11e49e25328f50392ee10b"} Jul 13 17:18:03.724876 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.724845 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" event={"ID":"e6d457d90ba56949705c8ce1904d5667","Type":"ContainerStarted","Data":"c709a2099f2a67cf98a52fc09327b6ee1412f63dc22cfdda8628ad01968907a6"} Jul 13 17:18:03.728171 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.728147 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bqkvc" event={"ID":"35d03ed5-a235-40aa-815f-b8f286ba77c3","Type":"ContainerStarted","Data":"1e7d549f3e383114b49a18a0b4b7ed059dfc494016aa3aa086d0efe315f21174"} Jul 13 17:18:03.734418 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.732483 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerStarted","Data":"d787bd27628d182f7fbf813718c9607478644a8bceaf3721cbd5e1f36f8c1128"} Jul 13 17:18:03.782354 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.782302 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-131.ec2.internal" podStartSLOduration=20.782287174 podStartE2EDuration="20.782287174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:03.746569176 +0000 UTC m=+21.903056051" watchObservedRunningTime="2026-07-13 17:18:03.782287174 +0000 UTC m=+21.938774047" Jul 13 17:18:03.808347 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.808277 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-131.ec2.internal" podStartSLOduration=20.808262456 podStartE2EDuration="20.808262456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:03.782816068 +0000 UTC m=+21.939302944" watchObservedRunningTime="2026-07-13 17:18:03.808262456 +0000 UTC m=+21.964749330" Jul 13 17:18:03.808722 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:03.808692 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bqkvc" podStartSLOduration=7.002010416 podStartE2EDuration="21.808685282s" totalImagesPullingTime="14.806674866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:03.808006416 +0000 UTC m=+21.964493289" watchObservedRunningTime="2026-07-13 17:18:03.808685282 +0000 UTC m=+21.965172156" Jul 13 17:18:04.736535 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.736497 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" event={"ID":"d3dc6950-ef2c-4fa2-b392-9483a8a3faed","Type":"ContainerStarted","Data":"3f400aabc927a520ed252f091e927ca884b649ac156a91e730a38244c5b40e5c"} Jul 13 17:18:04.738084 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.738062 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" event={"ID":"65942462-cf09-4171-8e03-8fa923e57a3b","Type":"ContainerStarted","Data":"5a9054a53cdb3d6897197c9cd9c1cf26ef3cc187def4c4fe51f0403c8e9ec6fa"} Jul 13 17:18:04.739599 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.739568 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z96wx" event={"ID":"13d2222c-9fec-4ae0-849a-49f4bf8b21e3","Type":"ContainerStarted","Data":"bee13e9b6c3d0f1488842672130dee67b65846c7e6f22fee7150f98d459f1243"} Jul 13 17:18:04.742410 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.742382 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:18:04.742837 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.742806 2579 generic.go:356] "Generic (PLEG): container finished" podID="ac6adcdd-a342-4652-8762-af45ca4d1114" containerID="24705b85a32e7c014b55b3a4ecdb848a71bae6aafbef8b041f5f004af600dfe9" exitCode=1 Jul 13 17:18:04.742914 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.742859 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"ff66397cd1c92e30ac42f0a740852bcdff792924a0e7b94a13f919697c0a9017"} Jul 13 17:18:04.742966 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.742945 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"beb5797e657ad28c4e37c5928feede399583a49cab267913748d6c5eeabcccb4"} Jul 13 17:18:04.743013 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.743001 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"b9a9175079b63b3f10dc7c742ad5340a02db353b3fd0328891596c800befa4a4"} Jul 13 17:18:04.743064 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.743050 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerDied","Data":"24705b85a32e7c014b55b3a4ecdb848a71bae6aafbef8b041f5f004af600dfe9"} Jul 13 17:18:04.743117 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.743104 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"606bd486b8fe5e15084dea1503b8e78212a416265223ecd4baaf6f43c70251a2"} Jul 13 17:18:04.744563 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.744414 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kl9vm" event={"ID":"cf7543c5-daf6-4ad6-9d05-d00958baac07","Type":"ContainerStarted","Data":"b6c62b9874ea69af9d7d28cd974db59f975eb553ffb1f2563daa1d3bd5be4f12"} Jul 13 17:18:04.746296 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.746233 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd72547b-0a43-40f8-8ec6-ea60f1634e41" containerID="d787bd27628d182f7fbf813718c9607478644a8bceaf3721cbd5e1f36f8c1128" exitCode=0 Jul 13 17:18:04.746401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.746307 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerDied","Data":"d787bd27628d182f7fbf813718c9607478644a8bceaf3721cbd5e1f36f8c1128"} Jul 13 17:18:04.748026 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.748006 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xch94" event={"ID":"3456e2f2-809f-4ccf-8b3d-3c0ddf53c583","Type":"ContainerStarted","Data":"3befdccb51c921acdc3d039c34c2e242e468b3c22b575e7785ad41b20fc090ae"} Jul 13 17:18:04.752382 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.752357 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j2dbs" event={"ID":"e411827e-84c9-4197-b166-8e0868a0709d","Type":"ContainerStarted","Data":"98ecf62f28d1834f068f313847c5fd8a1a864f431a2aaa7be7f24035fccb6735"} Jul 13 17:18:04.778344 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.778292 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h9tmj" podStartSLOduration=3.797327888 podStartE2EDuration="22.778284081s" totalImagesPullingTime="18.980956193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:04.776528881 +0000 UTC m=+22.933015754" watchObservedRunningTime="2026-07-13 17:18:04.778284081 +0000 UTC m=+22.934770968" Jul 13 17:18:04.862844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.861839 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xch94" podStartSLOduration=8.067742819 podStartE2EDuration="22.861820716s" totalImagesPullingTime="14.794077897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:04.861818825 +0000 UTC m=+23.018305701" watchObservedRunningTime="2026-07-13 17:18:04.861820716 +0000 UTC m=+23.018307590" Jul 13 17:18:04.917715 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.917675 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kl9vm" podStartSLOduration=2.940124002 podStartE2EDuration="21.91766338s" totalImagesPullingTime="18.977539378s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:04.917303699 +0000 UTC m=+23.073790573" watchObservedRunningTime="2026-07-13 17:18:04.91766338 +0000 UTC m=+23.074150253" Jul 13 17:18:04.917850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.917789 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-j2dbs" podStartSLOduration=2.91047494 podStartE2EDuration="21.917785649s" totalImagesPullingTime="19.007310709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:04.8960838 +0000 UTC m=+23.052570674" watchObservedRunningTime="2026-07-13 17:18:04.917785649 +0000 UTC m=+23.074272519" Jul 13 17:18:04.958421 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:04.958401 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:18:05.203053 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.203034 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 17:18:05.526144 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.526021 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T17:18:05.203048679Z","UUID":"bd049019-2fd0-4d2c-8dc4-7d6f65c46280","Handler":null,"Name":"","Endpoint":""} Jul 13 17:18:05.527529 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.527514 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 17:18:05.527626 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.527545 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 17:18:05.655450 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.655421 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:05.655450 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.655436 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:05.655683 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:05.655526 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:05.655683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.655533 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:05.655683 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:05.655643 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:05.655843 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:05.655720 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:05.757912 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.757881 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" event={"ID":"65942462-cf09-4171-8e03-8fa923e57a3b","Type":"ContainerStarted","Data":"cfa564964982490eed98ee4d742f46c300db9b08c9c9c4adefaf76083613fd44"} Jul 13 17:18:05.761632 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.761611 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:18:05.762123 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:05.762060 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"1c94ce936fca94bf995c7db6dc9f53e7727ca1dbd9274873571b9251039c7c1c"} Jul 13 17:18:06.766625 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:06.766423 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" event={"ID":"65942462-cf09-4171-8e03-8fa923e57a3b","Type":"ContainerStarted","Data":"975320352df2a39fa11576b191467e7ec135997073b30914671f44bc18756566"} Jul 13 17:18:06.798327 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:06.798285 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z96wx" podStartSLOduration=4.872045963 podStartE2EDuration="23.798266547s" totalImagesPullingTime="18.926220584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:04.95128595 +0000 UTC m=+23.107772825" watchObservedRunningTime="2026-07-13 17:18:06.798266547 +0000 UTC m=+24.954753415" Jul 13 17:18:07.655417 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:07.655376 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:07.655417 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:07.655424 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:07.655675 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:07.655491 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:07.655675 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:07.655558 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:07.655797 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:07.655670 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:07.655797 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:07.655778 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:07.772006 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:07.771981 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:18:07.772502 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:07.772395 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"b8844a74093e62a49d5dd0ab3b7fa9a93db0b057ba4955600fb27a92588324c6"} Jul 13 17:18:09.065726 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.065689 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:18:09.066340 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.066311 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:18:09.086192 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.086141 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6qlrw" podStartSLOduration=4.714320128 podStartE2EDuration="26.086122321s" totalImagesPullingTime="21.371802193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:06.798564802 +0000 UTC m=+24.955051693" watchObservedRunningTime="2026-07-13 17:18:09.086122321 +0000 UTC m=+27.242609200" Jul 13 17:18:09.656100 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.656069 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:09.656205 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.656157 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:09.656205 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:09.656162 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:09.656333 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.656166 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:09.656333 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:09.656270 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:09.656444 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:09.656365 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:09.780251 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.780100 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:18:09.780629 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.780602 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"4b07586e42d469a571011fb067ab256710f91a8ee517acccb79279d942840495"} Jul 13 17:18:09.781045 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.781028 2579 scope.go:117] "RemoveContainer" containerID="24705b85a32e7c014b55b3a4ecdb848a71bae6aafbef8b041f5f004af600dfe9" Jul 13 17:18:09.781473 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:09.781457 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z96wx" Jul 13 17:18:10.785654 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.785628 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:18:10.786257 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.786005 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" event={"ID":"ac6adcdd-a342-4652-8762-af45ca4d1114","Type":"ContainerStarted","Data":"14e19d602f85a4e919a87e61c1e1bffe137da02a19e0fd84f091e135666442c3"} Jul 13 17:18:10.786257 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.786226 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:18:10.786405 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.786302 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:18:10.786405 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.786360 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:18:10.788287 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.788256 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd72547b-0a43-40f8-8ec6-ea60f1634e41" containerID="e90e56907a14cf289a4ab1448d6b8f210fbb9c980e21f9efba7b19cb61bb9f1d" exitCode=0 Jul 13 17:18:10.788437 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.788337 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerDied","Data":"e90e56907a14cf289a4ab1448d6b8f210fbb9c980e21f9efba7b19cb61bb9f1d"} Jul 13 17:18:10.801552 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.801530 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:18:10.801648 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.801618 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:18:10.847215 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:10.847172 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" podStartSLOduration=8.264665593 podStartE2EDuration="27.847158762s" totalImagesPullingTime="19.582493169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:10.843677425 +0000 UTC m=+29.000164300" watchObservedRunningTime="2026-07-13 17:18:10.847158762 +0000 UTC m=+29.003645636" Jul 13 17:18:11.655455 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:11.655425 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:11.655591 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:11.655425 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:11.655591 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:11.655567 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:11.655660 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:11.655425 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:11.655660 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:11.655622 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:11.655725 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:11.655689 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:12.114969 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:12.114797 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-48dkb"] Jul 13 17:18:12.115355 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:12.115091 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:12.115355 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:12.115207 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:12.120727 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:12.120678 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-twrr7"] Jul 13 17:18:12.120871 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:12.120813 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:12.120932 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:12.120890 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:12.121266 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:12.121246 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f2p4s"] Jul 13 17:18:12.121404 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:12.121347 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:12.121448 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:12.121411 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:12.794628 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:12.794595 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd72547b-0a43-40f8-8ec6-ea60f1634e41" containerID="e29df69aac56b154937e2bcb2943f4ed215dfa944be5eb2a3deb86098065b0c4" exitCode=0 Jul 13 17:18:12.794822 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:12.794688 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerDied","Data":"e29df69aac56b154937e2bcb2943f4ed215dfa944be5eb2a3deb86098065b0c4"} Jul 13 17:18:13.655335 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:13.655292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:13.655335 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:13.655312 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:13.655970 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:13.655419 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:13.655970 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:13.655429 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:13.655970 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:13.655476 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:13.655970 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:13.655574 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:14.801671 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:14.801639 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd72547b-0a43-40f8-8ec6-ea60f1634e41" containerID="36678e47ace1df53fccaa6b3278221ba853824f398faa42946c86ac57961f3eb" exitCode=0 Jul 13 17:18:14.802125 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:14.801681 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerDied","Data":"36678e47ace1df53fccaa6b3278221ba853824f398faa42946c86ac57961f3eb"} Jul 13 17:18:15.655559 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:15.655499 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:15.655559 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:15.655546 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:15.655559 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:15.655543 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:15.655799 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:15.655661 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-twrr7" podUID="50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9" Jul 13 17:18:15.655799 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:15.655732 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:18:15.655893 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:15.655807 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f2p4s" podUID="721418c3-65df-44e7-8c2a-3f2395cfff5a" Jul 13 17:18:15.995126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:15.995093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:15.995637 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:15.995177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:15.995637 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:15.995245 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 17:18:15.995637 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:15.995274 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:18:15.995637 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:15.995337 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret podName:50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9 nodeName:}" failed. No retries permitted until 2026-07-13 17:18:47.995300094 +0000 UTC m=+66.151786962 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret") pod "global-pull-secret-syncer-twrr7" (UID: "50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9") : object "kube-system"/"original-pull-secret" not registered Jul 13 17:18:15.995637 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:15.995360 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:18:47.995350227 +0000 UTC m=+66.151837084 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 17:18:16.095978 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:16.095935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:16.096162 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:16.096114 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 17:18:16.096162 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:16.096142 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 17:18:16.096162 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:16.096154 2579 projected.go:196] Error preparing data for projected volume kube-api-access-98gzk for pod openshift-network-diagnostics/network-check-target-f2p4s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:18:16.096302 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:16.096204 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk podName:721418c3-65df-44e7-8c2a-3f2395cfff5a nodeName:}" failed. No retries permitted until 2026-07-13 17:18:48.096189685 +0000 UTC m=+66.252676538 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-98gzk" (UniqueName: "kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk") pod "network-check-target-f2p4s" (UID: "721418c3-65df-44e7-8c2a-3f2395cfff5a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 17:18:17.146931 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.146904 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-131.ec2.internal" event="NodeReady" Jul 13 17:18:17.147341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.147059 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 17:18:17.252829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.252187 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cdlrj"] Jul 13 17:18:17.254936 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.254912 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7qt69"] Jul 13 17:18:17.255043 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.255003 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.256610 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.256591 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7qt69" Jul 13 17:18:17.259390 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.259166 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 17:18:17.259746 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.259727 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 17:18:17.259746 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.259741 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 17:18:17.260038 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.260021 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 17:18:17.260114 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.260075 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 17:18:17.260213 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.260195 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vvlgt\"" Jul 13 17:18:17.265140 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.265118 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f7qlt\"" Jul 13 17:18:17.285106 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.285082 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7qt69"] Jul 13 17:18:17.292737 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.292714 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cdlrj"] Jul 13 17:18:17.405021 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.404981 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/535200a1-6053-4333-8463-07781e5f0486-cert\") pod \"ingress-canary-7qt69\" (UID: \"535200a1-6053-4333-8463-07781e5f0486\") " pod="openshift-ingress-canary/ingress-canary-7qt69" Jul 13 17:18:17.405197 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.405048 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9baab0f5-91c6-4a40-89b4-241bd5585973-config-volume\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.405197 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.405099 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9baab0f5-91c6-4a40-89b4-241bd5585973-tmp-dir\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.405197 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.405117 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gzj2\" (UniqueName: \"kubernetes.io/projected/9baab0f5-91c6-4a40-89b4-241bd5585973-kube-api-access-7gzj2\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.405352 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.405195 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.405352 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.405254 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhtv6\" (UniqueName: \"kubernetes.io/projected/535200a1-6053-4333-8463-07781e5f0486-kube-api-access-mhtv6\") pod \"ingress-canary-7qt69\" (UID: \"535200a1-6053-4333-8463-07781e5f0486\") " pod="openshift-ingress-canary/ingress-canary-7qt69" Jul 13 17:18:17.506231 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.506197 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/535200a1-6053-4333-8463-07781e5f0486-cert\") pod \"ingress-canary-7qt69\" (UID: \"535200a1-6053-4333-8463-07781e5f0486\") " pod="openshift-ingress-canary/ingress-canary-7qt69" Jul 13 17:18:17.506444 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.506254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9baab0f5-91c6-4a40-89b4-241bd5585973-config-volume\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.506444 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.506279 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9baab0f5-91c6-4a40-89b4-241bd5585973-tmp-dir\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.506444 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.506302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gzj2\" (UniqueName: \"kubernetes.io/projected/9baab0f5-91c6-4a40-89b4-241bd5585973-kube-api-access-7gzj2\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.506612 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.506435 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.506612 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.506480 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhtv6\" (UniqueName: \"kubernetes.io/projected/535200a1-6053-4333-8463-07781e5f0486-kube-api-access-mhtv6\") pod \"ingress-canary-7qt69\" (UID: \"535200a1-6053-4333-8463-07781e5f0486\") " pod="openshift-ingress-canary/ingress-canary-7qt69" Jul 13 17:18:17.506612 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:17.506568 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:18:17.506754 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:17.506631 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:18:18.006611184 +0000 UTC m=+36.163098037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:18:17.506754 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.506671 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9baab0f5-91c6-4a40-89b4-241bd5585973-tmp-dir\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.506976 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.506951 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9baab0f5-91c6-4a40-89b4-241bd5585973-config-volume\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.511229 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.511206 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/535200a1-6053-4333-8463-07781e5f0486-cert\") pod \"ingress-canary-7qt69\" (UID: \"535200a1-6053-4333-8463-07781e5f0486\") " pod="openshift-ingress-canary/ingress-canary-7qt69" Jul 13 17:18:17.525883 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.525860 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gzj2\" (UniqueName: \"kubernetes.io/projected/9baab0f5-91c6-4a40-89b4-241bd5585973-kube-api-access-7gzj2\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:17.526427 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.526408 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhtv6\" (UniqueName: \"kubernetes.io/projected/535200a1-6053-4333-8463-07781e5f0486-kube-api-access-mhtv6\") pod \"ingress-canary-7qt69\" (UID: \"535200a1-6053-4333-8463-07781e5f0486\") " pod="openshift-ingress-canary/ingress-canary-7qt69" Jul 13 17:18:17.573487 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.573459 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7qt69" Jul 13 17:18:17.655595 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.655481 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:17.655762 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.655481 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:17.655819 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.655481 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:17.663878 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.663645 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 17:18:17.663878 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.663675 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 17:18:17.663878 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.663714 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 17:18:17.663878 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.663798 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gkqtt\"" Jul 13 17:18:17.663878 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.663798 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-56qtx\"" Jul 13 17:18:17.664774 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:17.664756 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 17:18:18.010190 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:18.010110 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:18.010376 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:18.010234 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:18:18.010376 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:18.010297 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:18:19.010281686 +0000 UTC m=+37.166768544 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:18:18.929972 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:18.929944 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7qt69"] Jul 13 17:18:18.935004 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:18:18.934983 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod535200a1_6053_4333_8463_07781e5f0486.slice/crio-203a784bb48f589bfb64aa5f565df2a798bb79c2abe938e85b4cb4af29100ecf WatchSource:0}: Error finding container 203a784bb48f589bfb64aa5f565df2a798bb79c2abe938e85b4cb4af29100ecf: Status 404 returned error can't find the container with id 203a784bb48f589bfb64aa5f565df2a798bb79c2abe938e85b4cb4af29100ecf Jul 13 17:18:19.017786 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:19.017753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:19.017961 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:19.017940 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:18:19.018045 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:19.018034 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:18:21.018012772 +0000 UTC m=+39.174499628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:18:19.817995 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:19.817765 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd72547b-0a43-40f8-8ec6-ea60f1634e41" containerID="39a719fc6f6b8b800ff0ecb576260ee4dd83e64c9e6008bad21a3ee44df12292" exitCode=0 Jul 13 17:18:19.818175 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:19.817858 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerDied","Data":"39a719fc6f6b8b800ff0ecb576260ee4dd83e64c9e6008bad21a3ee44df12292"} Jul 13 17:18:19.819299 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:19.819272 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7qt69" event={"ID":"535200a1-6053-4333-8463-07781e5f0486","Type":"ContainerStarted","Data":"203a784bb48f589bfb64aa5f565df2a798bb79c2abe938e85b4cb4af29100ecf"} Jul 13 17:18:20.825469 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:20.825438 2579 generic.go:356] "Generic (PLEG): container finished" podID="bd72547b-0a43-40f8-8ec6-ea60f1634e41" containerID="8adb7d98fa6423c5318d13fc1f855d8fe203e94db069a091627141d44c7ff80c" exitCode=0 Jul 13 17:18:20.825955 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:20.825493 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerDied","Data":"8adb7d98fa6423c5318d13fc1f855d8fe203e94db069a091627141d44c7ff80c"} Jul 13 17:18:21.031416 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:21.031371 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:21.031597 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:21.031552 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:18:21.031660 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:21.031633 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:18:25.031610601 +0000 UTC m=+43.188097459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:18:21.831665 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:21.831577 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" event={"ID":"bd72547b-0a43-40f8-8ec6-ea60f1634e41","Type":"ContainerStarted","Data":"3d6702114a29bd300c578d706e6094abb2be4fb41eb6e18e4817c38d0e8334c8"} Jul 13 17:18:21.833015 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:21.832992 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7qt69" event={"ID":"535200a1-6053-4333-8463-07781e5f0486","Type":"ContainerStarted","Data":"3c163b8d8b1b4c9c5c2e2230c72e06efd653f24b4816f4663f5db78e54f779a2"} Jul 13 17:18:21.872989 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:21.872946 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cwbb8" podStartSLOduration=8.679777984 podStartE2EDuration="38.872931941s" totalImagesPullingTime="30.193153957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:21.872096806 +0000 UTC m=+40.028583692" watchObservedRunningTime="2026-07-13 17:18:21.872931941 +0000 UTC m=+40.029418816" Jul 13 17:18:21.900393 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:21.900350 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7qt69" podStartSLOduration=2.383090003 podStartE2EDuration="4.900316665s" totalImagesPullingTime="2.517226662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:21.896673112 +0000 UTC m=+40.053159988" watchObservedRunningTime="2026-07-13 17:18:21.900316665 +0000 UTC m=+40.056803539" Jul 13 17:18:25.058939 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:25.058893 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:25.059301 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:25.059041 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:18:25.059301 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:25.059098 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:18:33.05908365 +0000 UTC m=+51.215570503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:18:33.108395 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:33.108350 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:33.108878 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:33.108484 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:18:33.108878 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:33.108548 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:18:49.108533026 +0000 UTC m=+67.265019878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:18:42.812097 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:42.812061 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-snh5v" Jul 13 17:18:48.016752 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.016710 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:18:48.017198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.016767 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:48.021003 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.020985 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 17:18:48.021094 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.021076 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 17:18:48.027662 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:48.027644 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 17:18:48.027763 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:48.027752 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:19:52.027734123 +0000 UTC m=+130.184220976 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : secret "metrics-daemon-secret" not found Jul 13 17:18:48.030779 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.030758 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9-original-pull-secret\") pod \"global-pull-secret-syncer-twrr7\" (UID: \"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9\") " pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:48.118053 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.118012 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:48.122717 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.122699 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 17:18:48.135076 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.135055 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 17:18:48.141363 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.141344 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98gzk\" (UniqueName: \"kubernetes.io/projected/721418c3-65df-44e7-8c2a-3f2395cfff5a-kube-api-access-98gzk\") pod \"network-check-target-f2p4s\" (UID: \"721418c3-65df-44e7-8c2a-3f2395cfff5a\") " pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:48.267990 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.267896 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-twrr7" Jul 13 17:18:48.278109 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.278085 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-56qtx\"" Jul 13 17:18:48.284282 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.284263 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:48.460634 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.460606 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-twrr7"] Jul 13 17:18:48.463685 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:18:48.463658 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f4cbd8_b73c_4a9f_8e83_4e192a9fc5d9.slice/crio-362ac5e5b58059eab8cca7f0f780822d1f950d135d0e332fdd6a0c1b6a45c029 WatchSource:0}: Error finding container 362ac5e5b58059eab8cca7f0f780822d1f950d135d0e332fdd6a0c1b6a45c029: Status 404 returned error can't find the container with id 362ac5e5b58059eab8cca7f0f780822d1f950d135d0e332fdd6a0c1b6a45c029 Jul 13 17:18:48.478046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.478027 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f2p4s"] Jul 13 17:18:48.479501 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:18:48.479476 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod721418c3_65df_44e7_8c2a_3f2395cfff5a.slice/crio-f9109bccde196f131e262d4b8eee35c8aa33189b5ee3a05f53943105196f57cf WatchSource:0}: Error finding container f9109bccde196f131e262d4b8eee35c8aa33189b5ee3a05f53943105196f57cf: Status 404 returned error can't find the container with id f9109bccde196f131e262d4b8eee35c8aa33189b5ee3a05f53943105196f57cf Jul 13 17:18:48.897075 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.897036 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f2p4s" event={"ID":"721418c3-65df-44e7-8c2a-3f2395cfff5a","Type":"ContainerStarted","Data":"f9109bccde196f131e262d4b8eee35c8aa33189b5ee3a05f53943105196f57cf"} Jul 13 17:18:48.898154 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:48.898125 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-twrr7" event={"ID":"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9","Type":"ContainerStarted","Data":"362ac5e5b58059eab8cca7f0f780822d1f950d135d0e332fdd6a0c1b6a45c029"} Jul 13 17:18:49.125258 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:49.125222 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:18:49.125788 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:49.125363 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:18:49.125788 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:18:49.125417 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:19:21.125403201 +0000 UTC m=+99.281890055 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:18:53.912899 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:53.912855 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f2p4s" event={"ID":"721418c3-65df-44e7-8c2a-3f2395cfff5a","Type":"ContainerStarted","Data":"e99076f9a743fb060b0fd72b24d6692de992ad66679d541af667522b3b53eb07"} Jul 13 17:18:53.913342 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:53.912925 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:18:53.914316 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:53.914293 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-twrr7" event={"ID":"50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9","Type":"ContainerStarted","Data":"c85f35c2d49cb3ff63a7f8ddd158b4332dc0046ff78762d81baf984a0d9e8842"} Jul 13 17:18:53.945911 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:18:53.945868 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f2p4s" podStartSLOduration=66.31269293 podStartE2EDuration="1m10.945856806s" totalImagesPullingTime="4.633163876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:53.945506133 +0000 UTC m=+72.101993010" watchObservedRunningTime="2026-07-13 17:18:53.945856806 +0000 UTC m=+72.102343680" Jul 13 17:19:21.148223 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:19:21.148096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:19:21.148719 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:19:21.148260 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:19:21.148719 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:19:21.148359 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:20:25.148340294 +0000 UTC m=+163.304827147 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:19:24.918774 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:19:24.918742 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f2p4s" Jul 13 17:19:24.941417 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:19:24.941346 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-twrr7" podStartSLOduration=97.288907259 podStartE2EDuration="1m41.941286183s" totalImagesPullingTime="4.652378924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:53.989014626 +0000 UTC m=+72.145501502" watchObservedRunningTime="2026-07-13 17:19:24.941286183 +0000 UTC m=+103.097773059" Jul 13 17:19:52.056206 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:19:52.056169 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:19:52.056698 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:19:52.056290 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 13 17:19:52.056698 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:19:52.056376 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs podName:87c1ac36-6870-4e28-97a9-1df7abf70a5e nodeName:}" failed. No retries permitted until 2026-07-13 17:21:54.05635957 +0000 UTC m=+252.212846429 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs") pod "network-metrics-daemon-48dkb" (UID: "87c1ac36-6870-4e28-97a9-1df7abf70a5e") : secret "metrics-daemon-secret" not found Jul 13 17:20:20.267606 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:20:20.267549 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cdlrj" podUID="9baab0f5-91c6-4a40-89b4-241bd5585973" Jul 13 17:20:20.678304 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:20:20.678257 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-48dkb" podUID="87c1ac36-6870-4e28-97a9-1df7abf70a5e" Jul 13 17:20:21.121388 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:20:21.121303 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cdlrj" Jul 13 17:20:25.169488 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:20:25.169442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:20:25.170007 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:20:25.169574 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 17:20:25.170007 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:20:25.169633 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls podName:9baab0f5-91c6-4a40-89b4-241bd5585973 nodeName:}" failed. No retries permitted until 2026-07-13 17:22:27.169618845 +0000 UTC m=+285.326105699 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls") pod "dns-default-cdlrj" (UID: "9baab0f5-91c6-4a40-89b4-241bd5585973") : secret "dns-default-metrics-tls" not found Jul 13 17:20:32.656422 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:20:32.656384 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:21:24.836276 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.836224 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2"] Jul 13 17:21:24.839148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.839102 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-56768b9b-6x2s5"] Jul 13 17:21:24.839283 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.839244 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:24.841991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.841970 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:24.846371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.846138 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 13 17:21:24.846450 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.846374 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pjqtp\"" Jul 13 17:21:24.846514 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.846498 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 17:21:24.847415 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.847384 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 17:21:24.847415 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.847406 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 17:21:24.847652 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.847449 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 13 17:21:24.847871 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.847838 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 13 17:21:24.847957 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.847930 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 13 17:21:24.848106 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.848088 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 13 17:21:24.850211 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.850194 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hrxbt\"" Jul 13 17:21:24.856138 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.856119 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 13 17:21:24.856857 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.856840 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 17:21:24.862554 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.862532 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2"] Jul 13 17:21:24.869581 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.869559 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56768b9b-6x2s5"] Jul 13 17:21:24.922177 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.922159 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j"] Jul 13 17:21:24.924844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.924828 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j" Jul 13 17:21:24.927751 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.927728 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:21:24.927751 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.927741 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 13 17:21:24.927751 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.927749 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tbwl8\"" Jul 13 17:21:24.931028 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.931007 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hql5g"] Jul 13 17:21:24.933657 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.933641 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2"] Jul 13 17:21:24.933761 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.933747 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:24.936581 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.936564 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:24.939981 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.939951 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j"] Jul 13 17:21:24.940074 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940008 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:21:24.940235 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940209 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 13 17:21:24.940341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940215 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 17:21:24.940341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940276 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pnp7d\"" Jul 13 17:21:24.940341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940302 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 13 17:21:24.940341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940216 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 13 17:21:24.940550 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940446 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 17:21:24.940550 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940475 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-dtzsl\"" Jul 13 17:21:24.940550 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.940484 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 13 17:21:24.954684 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.954659 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h5tl\" (UniqueName: \"kubernetes.io/projected/6714944a-56b5-4b88-b10e-22fd57415282-kube-api-access-2h5tl\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:24.954771 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.954706 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:24.954771 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.954755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-default-certificate\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:24.954860 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.954781 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:24.954860 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.954807 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvtct\" (UniqueName: \"kubernetes.io/projected/cc37ddc9-06d8-4427-b437-aff8b7169805-kube-api-access-pvtct\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:24.954860 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.954850 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc37ddc9-06d8-4427-b437-aff8b7169805-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:24.954983 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.954874 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:24.954983 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.954906 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-stats-auth\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:24.956129 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.956111 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hql5g"] Jul 13 17:21:24.963211 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.963190 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2"] Jul 13 17:21:24.972581 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:24.972565 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 13 17:21:25.055792 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.055769 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a9856f9-2976-466d-b07f-e43582a7d59a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.055903 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.055811 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc2vf\" (UniqueName: \"kubernetes.io/projected/6054271a-bcd5-495c-b204-70feb3e81753-kube-api-access-fc2vf\") pod \"volume-data-source-validator-78495597f6-sfp7j\" (UID: \"6054271a-bcd5-495c-b204-70feb3e81753\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j" Jul 13 17:21:25.055903 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.055837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a9856f9-2976-466d-b07f-e43582a7d59a-serving-cert\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.055903 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.055873 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc37ddc9-06d8-4427-b437-aff8b7169805-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:25.056061 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.055926 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:25.056061 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056008 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-226sg\" (UniqueName: \"kubernetes.io/projected/7a9856f9-2976-466d-b07f-e43582a7d59a-kube-api-access-226sg\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.056061 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.056012 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:25.056061 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056048 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-stats-auth\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.056253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w4sl\" (UniqueName: \"kubernetes.io/projected/27bf791f-9d99-481e-8b1b-bc504cbeea9d-kube-api-access-6w4sl\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:25.056253 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.056082 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls podName:cc37ddc9-06d8-4427-b437-aff8b7169805 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:25.556062511 +0000 UTC m=+223.712549366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6nfw2" (UID: "cc37ddc9-06d8-4427-b437-aff8b7169805") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:25.056253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056162 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7a9856f9-2976-466d-b07f-e43582a7d59a-snapshots\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.056253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056201 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h5tl\" (UniqueName: \"kubernetes.io/projected/6714944a-56b5-4b88-b10e-22fd57415282-kube-api-access-2h5tl\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.056253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056308 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a9856f9-2976-466d-b07f-e43582a7d59a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.056357 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.056404 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:25.556387976 +0000 UTC m=+223.712874839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : secret "router-metrics-certs-default" not found Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056393 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-default-certificate\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056442 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvtct\" (UniqueName: \"kubernetes.io/projected/cc37ddc9-06d8-4427-b437-aff8b7169805-kube-api-access-pvtct\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a9856f9-2976-466d-b07f-e43582a7d59a-tmp\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.056546 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.056532 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:25.556520516 +0000 UTC m=+223.713007375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : configmap references non-existent config key: service-ca.crt Jul 13 17:21:25.056889 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.056659 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc37ddc9-06d8-4427-b437-aff8b7169805-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:25.058302 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.058285 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-stats-auth\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.058580 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.058561 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-default-certificate\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.074358 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.071136 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h5tl\" (UniqueName: \"kubernetes.io/projected/6714944a-56b5-4b88-b10e-22fd57415282-kube-api-access-2h5tl\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.076980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.076962 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvtct\" (UniqueName: \"kubernetes.io/projected/cc37ddc9-06d8-4427-b437-aff8b7169805-kube-api-access-pvtct\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:25.157170 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157149 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-226sg\" (UniqueName: \"kubernetes.io/projected/7a9856f9-2976-466d-b07f-e43582a7d59a-kube-api-access-226sg\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.157252 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w4sl\" (UniqueName: \"kubernetes.io/projected/27bf791f-9d99-481e-8b1b-bc504cbeea9d-kube-api-access-6w4sl\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:25.157252 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157200 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7a9856f9-2976-466d-b07f-e43582a7d59a-snapshots\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.157355 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157335 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:25.157413 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157398 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a9856f9-2976-466d-b07f-e43582a7d59a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.157479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a9856f9-2976-466d-b07f-e43582a7d59a-tmp\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.157516 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157497 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a9856f9-2976-466d-b07f-e43582a7d59a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.157516 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.157502 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:21:25.157584 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157537 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc2vf\" (UniqueName: \"kubernetes.io/projected/6054271a-bcd5-495c-b204-70feb3e81753-kube-api-access-fc2vf\") pod \"volume-data-source-validator-78495597f6-sfp7j\" (UID: \"6054271a-bcd5-495c-b204-70feb3e81753\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j" Jul 13 17:21:25.157584 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.157560 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls podName:27bf791f-9d99-481e-8b1b-bc504cbeea9d nodeName:}" failed. No retries permitted until 2026-07-13 17:21:25.657546897 +0000 UTC m=+223.814033750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-gqhb2" (UID: "27bf791f-9d99-481e-8b1b-bc504cbeea9d") : secret "samples-operator-tls" not found Jul 13 17:21:25.157584 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a9856f9-2976-466d-b07f-e43582a7d59a-serving-cert\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.157956 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157937 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a9856f9-2976-466d-b07f-e43582a7d59a-tmp\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.158035 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.157975 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7a9856f9-2976-466d-b07f-e43582a7d59a-snapshots\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.158035 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.158013 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a9856f9-2976-466d-b07f-e43582a7d59a-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.158557 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.158538 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a9856f9-2976-466d-b07f-e43582a7d59a-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.159604 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.159583 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a9856f9-2976-466d-b07f-e43582a7d59a-serving-cert\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.166170 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.166151 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-226sg\" (UniqueName: \"kubernetes.io/projected/7a9856f9-2976-466d-b07f-e43582a7d59a-kube-api-access-226sg\") pod \"insights-operator-68d9dd94cf-hql5g\" (UID: \"7a9856f9-2976-466d-b07f-e43582a7d59a\") " pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.166541 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.166521 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w4sl\" (UniqueName: \"kubernetes.io/projected/27bf791f-9d99-481e-8b1b-bc504cbeea9d-kube-api-access-6w4sl\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:25.166593 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.166564 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc2vf\" (UniqueName: \"kubernetes.io/projected/6054271a-bcd5-495c-b204-70feb3e81753-kube-api-access-fc2vf\") pod \"volume-data-source-validator-78495597f6-sfp7j\" (UID: \"6054271a-bcd5-495c-b204-70feb3e81753\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j" Jul 13 17:21:25.233607 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.233577 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j" Jul 13 17:21:25.243749 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.243734 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" Jul 13 17:21:25.365001 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.364974 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j"] Jul 13 17:21:25.368006 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:25.367983 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6054271a_bcd5_495c_b204_70feb3e81753.slice/crio-005b6a0f2ee1d3c5b7231b68005988a0cc44c0f12fcc42bc8ed59761d69b4886 WatchSource:0}: Error finding container 005b6a0f2ee1d3c5b7231b68005988a0cc44c0f12fcc42bc8ed59761d69b4886: Status 404 returned error can't find the container with id 005b6a0f2ee1d3c5b7231b68005988a0cc44c0f12fcc42bc8ed59761d69b4886 Jul 13 17:21:25.383267 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.383203 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hql5g"] Jul 13 17:21:25.386147 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:25.386126 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9856f9_2976_466d_b07f_e43582a7d59a.slice/crio-785df6b494bd481cfaf55b0b641524b0dcb1e5faa87e0ef28efbc3f524dcf6e7 WatchSource:0}: Error finding container 785df6b494bd481cfaf55b0b641524b0dcb1e5faa87e0ef28efbc3f524dcf6e7: Status 404 returned error can't find the container with id 785df6b494bd481cfaf55b0b641524b0dcb1e5faa87e0ef28efbc3f524dcf6e7 Jul 13 17:21:25.560931 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.560860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.560931 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.560909 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:25.561096 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.561054 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:26.56103042 +0000 UTC m=+224.717517304 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : configmap references non-existent config key: service-ca.crt Jul 13 17:21:25.561165 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.561106 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:25.561165 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.561160 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls podName:cc37ddc9-06d8-4427-b437-aff8b7169805 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:26.56114725 +0000 UTC m=+224.717634103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6nfw2" (UID: "cc37ddc9-06d8-4427-b437-aff8b7169805") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:25.561272 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.561113 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:25.561272 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.561194 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:21:25.561272 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.561248 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:26.561232884 +0000 UTC m=+224.717719796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : secret "router-metrics-certs-default" not found Jul 13 17:21:25.662182 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:25.662153 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:25.662298 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.662278 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:21:25.662368 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:25.662351 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls podName:27bf791f-9d99-481e-8b1b-bc504cbeea9d nodeName:}" failed. No retries permitted until 2026-07-13 17:21:26.662339472 +0000 UTC m=+224.818826325 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-gqhb2" (UID: "27bf791f-9d99-481e-8b1b-bc504cbeea9d") : secret "samples-operator-tls" not found Jul 13 17:21:26.267095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:26.267057 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" event={"ID":"7a9856f9-2976-466d-b07f-e43582a7d59a","Type":"ContainerStarted","Data":"785df6b494bd481cfaf55b0b641524b0dcb1e5faa87e0ef28efbc3f524dcf6e7"} Jul 13 17:21:26.268029 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:26.268008 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j" event={"ID":"6054271a-bcd5-495c-b204-70feb3e81753","Type":"ContainerStarted","Data":"005b6a0f2ee1d3c5b7231b68005988a0cc44c0f12fcc42bc8ed59761d69b4886"} Jul 13 17:21:26.568830 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:26.568725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:26.568830 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:26.568805 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:26.569041 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:26.568862 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:26.569041 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:26.568952 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:21:26.569041 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:26.568994 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:26.569041 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:26.569027 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:28.569004527 +0000 UTC m=+226.725491395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : secret "router-metrics-certs-default" not found Jul 13 17:21:26.569241 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:26.569046 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:28.569037486 +0000 UTC m=+226.725524339 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : configmap references non-existent config key: service-ca.crt Jul 13 17:21:26.569241 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:26.569067 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls podName:cc37ddc9-06d8-4427-b437-aff8b7169805 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:28.56905413 +0000 UTC m=+226.725540992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6nfw2" (UID: "cc37ddc9-06d8-4427-b437-aff8b7169805") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:26.670174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:26.670122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:26.670377 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:26.670315 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:21:26.670453 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:26.670406 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls podName:27bf791f-9d99-481e-8b1b-bc504cbeea9d nodeName:}" failed. No retries permitted until 2026-07-13 17:21:28.670386343 +0000 UTC m=+226.826873198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-gqhb2" (UID: "27bf791f-9d99-481e-8b1b-bc504cbeea9d") : secret "samples-operator-tls" not found Jul 13 17:21:27.272058 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:27.272020 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j" event={"ID":"6054271a-bcd5-495c-b204-70feb3e81753","Type":"ContainerStarted","Data":"7a25948de95d90c05c805c5fb761104b9ee65b5f3eceb1939011bfc505fc8ba6"} Jul 13 17:21:28.278916 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.278881 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" event={"ID":"7a9856f9-2976-466d-b07f-e43582a7d59a","Type":"ContainerStarted","Data":"91a483bdd6506901564aea205335a43a8a671e2035ce3b2054424037efb0acce"} Jul 13 17:21:28.297703 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.297641 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" podStartSLOduration=2.048331622 podStartE2EDuration="4.297619917s" totalImagesPullingTime="2.249288295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:21:28.296997763 +0000 UTC m=+226.453484640" watchObservedRunningTime="2026-07-13 17:21:28.297619917 +0000 UTC m=+226.454106822" Jul 13 17:21:28.297984 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.297961 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-sfp7j" podStartSLOduration=2.933917303 podStartE2EDuration="4.297952149s" totalImagesPullingTime="1.364034846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:21:27.287750622 +0000 UTC m=+225.444237496" watchObservedRunningTime="2026-07-13 17:21:28.297952149 +0000 UTC m=+226.454439058" Jul 13 17:21:28.583900 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.583794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:28.583900 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.583877 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:28.583900 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.583912 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:28.584175 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:28.583945 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:28.584175 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:28.584010 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:21:28.584175 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:28.584012 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls podName:cc37ddc9-06d8-4427-b437-aff8b7169805 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:32.583997261 +0000 UTC m=+230.740484115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6nfw2" (UID: "cc37ddc9-06d8-4427-b437-aff8b7169805") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:28.584175 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:28.584064 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:32.584052048 +0000 UTC m=+230.740538900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : configmap references non-existent config key: service-ca.crt Jul 13 17:21:28.584175 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:28.584075 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:32.584068958 +0000 UTC m=+230.740555811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : secret "router-metrics-certs-default" not found Jul 13 17:21:28.685115 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.685077 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:28.685276 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:28.685213 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:21:28.685276 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:28.685274 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls podName:27bf791f-9d99-481e-8b1b-bc504cbeea9d nodeName:}" failed. No retries permitted until 2026-07-13 17:21:32.685259212 +0000 UTC m=+230.841746065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-gqhb2" (UID: "27bf791f-9d99-481e-8b1b-bc504cbeea9d") : secret "samples-operator-tls" not found Jul 13 17:21:28.878168 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.878135 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-289jk"] Jul 13 17:21:28.881443 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.881421 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:28.885919 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.885890 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 17:21:28.886777 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.886577 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 17:21:28.886777 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.886591 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m4nm5\"" Jul 13 17:21:28.938811 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.938777 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-289jk"] Jul 13 17:21:28.987715 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.987680 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-869lg\" (UniqueName: \"kubernetes.io/projected/4caee1b8-5d63-4814-b089-967a32b2e038-kube-api-access-869lg\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:28.987715 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.987717 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:28.987905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.987735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4caee1b8-5d63-4814-b089-967a32b2e038-crio-socket\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:28.987905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.987819 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4caee1b8-5d63-4814-b089-967a32b2e038-data-volume\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:28.987905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:28.987864 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.089547 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-869lg\" (UniqueName: \"kubernetes.io/projected/4caee1b8-5d63-4814-b089-967a32b2e038-kube-api-access-869lg\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.089615 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.089659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4caee1b8-5d63-4814-b089-967a32b2e038-crio-socket\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.089734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4caee1b8-5d63-4814-b089-967a32b2e038-data-volume\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.089764 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:29.090001 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:29.090072 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:29.590054323 +0000 UTC m=+227.746541180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : secret "insights-runtime-extractor-tls" not found Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:29.090673 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:29.090728 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:29.590706272 +0000 UTC m=+227.747193124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : configmap "kube-rbac-proxy" not found Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.090799 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4caee1b8-5d63-4814-b089-967a32b2e038-crio-socket\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.092543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.091116 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4caee1b8-5d63-4814-b089-967a32b2e038-data-volume\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.098366 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.098343 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-869lg\" (UniqueName: \"kubernetes.io/projected/4caee1b8-5d63-4814-b089-967a32b2e038-kube-api-access-869lg\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.593887 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.593850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.594261 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.593910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:29.594261 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:29.593975 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:21:29.594261 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:29.594020 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:21:29.594261 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:29.594071 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:30.594056661 +0000 UTC m=+228.750543514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : secret "insights-runtime-extractor-tls" not found Jul 13 17:21:29.594261 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:29.594083 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:30.594076941 +0000 UTC m=+228.750563794 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : configmap "kube-rbac-proxy" not found Jul 13 17:21:29.946143 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.946110 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb"] Jul 13 17:21:29.949300 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.949282 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" Jul 13 17:21:29.951365 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.951348 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 13 17:21:29.951628 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.951613 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 13 17:21:29.952213 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.952197 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-swhp6\"" Jul 13 17:21:29.958266 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.958244 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb"] Jul 13 17:21:29.997147 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:29.997123 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7sh4\" (UniqueName: \"kubernetes.io/projected/3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e-kube-api-access-w7sh4\") pod \"migrator-7f59b54f6c-qf4wb\" (UID: \"3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" Jul 13 17:21:30.097836 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:30.097807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7sh4\" (UniqueName: \"kubernetes.io/projected/3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e-kube-api-access-w7sh4\") pod \"migrator-7f59b54f6c-qf4wb\" (UID: \"3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" Jul 13 17:21:30.107890 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:30.107866 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7sh4\" (UniqueName: \"kubernetes.io/projected/3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e-kube-api-access-w7sh4\") pod \"migrator-7f59b54f6c-qf4wb\" (UID: \"3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" Jul 13 17:21:30.258622 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:30.258536 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" Jul 13 17:21:30.380679 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:30.380642 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb"] Jul 13 17:21:30.383837 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:30.383811 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0dfe0e_61f4_4f6f_8e1d_c8274642cd2e.slice/crio-deb3b20663326d50e818e77a9c1dc030e03e17ae0f060dcf3fecf87ced040985 WatchSource:0}: Error finding container deb3b20663326d50e818e77a9c1dc030e03e17ae0f060dcf3fecf87ced040985: Status 404 returned error can't find the container with id deb3b20663326d50e818e77a9c1dc030e03e17ae0f060dcf3fecf87ced040985 Jul 13 17:21:30.601573 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:30.601516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:30.601573 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:30.601568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:30.601943 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:30.601656 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:21:30.601943 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:30.601678 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:21:30.601943 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:30.601730 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:32.601708866 +0000 UTC m=+230.758195718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : configmap "kube-rbac-proxy" not found Jul 13 17:21:30.601943 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:30.601754 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:32.60174296 +0000 UTC m=+230.758229829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : secret "insights-runtime-extractor-tls" not found Jul 13 17:21:30.623270 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:30.623241 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kl9vm_cf7543c5-daf6-4ad6-9d05-d00958baac07/dns-node-resolver/0.log" Jul 13 17:21:31.288616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:31.288581 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" event={"ID":"3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e","Type":"ContainerStarted","Data":"deb3b20663326d50e818e77a9c1dc030e03e17ae0f060dcf3fecf87ced040985"} Jul 13 17:21:31.623652 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:31.623585 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bqkvc_35d03ed5-a235-40aa-815f-b8f286ba77c3/node-ca/0.log" Jul 13 17:21:32.292607 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.292573 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" event={"ID":"3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e","Type":"ContainerStarted","Data":"758f734fa4e1535d3f99b96e3c4ade46b073ada637c0629b3fe551e351488ca2"} Jul 13 17:21:32.292772 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.292631 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" event={"ID":"3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e","Type":"ContainerStarted","Data":"aeb38d36acb6736a1f720ab8422b10d563aaa6d65db173cb514cc4da6155067f"} Jul 13 17:21:32.312160 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.312117 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-qf4wb" podStartSLOduration=2.067315707 podStartE2EDuration="3.312103653s" totalImagesPullingTime="1.244787946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:21:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:21:32.311838937 +0000 UTC m=+230.468325813" watchObservedRunningTime="2026-07-13 17:21:32.312103653 +0000 UTC m=+230.468590527" Jul 13 17:21:32.423212 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.423187 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7qt69_535200a1-6053-4333-8463-07781e5f0486/serve-healthcheck-canary/0.log" Jul 13 17:21:32.617909 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.617834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:32.617909 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.617872 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:32.618095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.617910 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:32.618095 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.617962 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 17:21:32.618095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.617964 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:32.618095 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.618008 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:40.617994897 +0000 UTC m=+238.774481750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : secret "router-metrics-certs-default" not found Jul 13 17:21:32.618095 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.618037 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:21:32.618095 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.618042 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:21:32.618095 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.618044 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle podName:6714944a-56b5-4b88-b10e-22fd57415282 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:40.618030554 +0000 UTC m=+238.774517406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle") pod "router-default-56768b9b-6x2s5" (UID: "6714944a-56b5-4b88-b10e-22fd57415282") : configmap references non-existent config key: service-ca.crt Jul 13 17:21:32.618095 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.618095 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:36.618084326 +0000 UTC m=+234.774571179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : secret "insights-runtime-extractor-tls" not found Jul 13 17:21:32.618383 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.618110 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:36.61810288 +0000 UTC m=+234.774589733 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : configmap "kube-rbac-proxy" not found Jul 13 17:21:32.618383 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.618145 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:32.618383 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.618228 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:32.618383 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.618256 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls podName:cc37ddc9-06d8-4427-b437-aff8b7169805 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:40.618249784 +0000 UTC m=+238.774736637 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6nfw2" (UID: "cc37ddc9-06d8-4427-b437-aff8b7169805") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:32.719372 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.719342 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 17:21:32.719745 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:32.719408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:32.719745 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:32.719627 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls podName:27bf791f-9d99-481e-8b1b-bc504cbeea9d nodeName:}" failed. No retries permitted until 2026-07-13 17:21:40.719609483 +0000 UTC m=+238.876096341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls") pod "cluster-samples-operator-65db858894-gqhb2" (UID: "27bf791f-9d99-481e-8b1b-bc504cbeea9d") : secret "samples-operator-tls" not found Jul 13 17:21:36.649984 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:36.649954 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:36.650357 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:36.650008 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:36.650357 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:36.650070 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 17:21:36.650357 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:36.650130 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 17:21:36.650357 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:36.650136 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:44.650118483 +0000 UTC m=+242.806605339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : configmap "kube-rbac-proxy" not found Jul 13 17:21:36.650357 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:36.650174 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls podName:4caee1b8-5d63-4814-b089-967a32b2e038 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:44.650164095 +0000 UTC m=+242.806650949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls") pod "insights-runtime-extractor-289jk" (UID: "4caee1b8-5d63-4814-b089-967a32b2e038") : secret "insights-runtime-extractor-tls" not found Jul 13 17:21:40.679155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.679118 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:40.679577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.679163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:40.679577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.679260 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:40.679577 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:40.679397 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:40.679577 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:21:40.679468 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls podName:cc37ddc9-06d8-4427-b437-aff8b7169805 nodeName:}" failed. No retries permitted until 2026-07-13 17:21:56.679448042 +0000 UTC m=+254.835934901 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6nfw2" (UID: "cc37ddc9-06d8-4427-b437-aff8b7169805") : secret "cluster-monitoring-operator-tls" not found Jul 13 17:21:40.679901 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.679883 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6714944a-56b5-4b88-b10e-22fd57415282-service-ca-bundle\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:40.681450 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.681430 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6714944a-56b5-4b88-b10e-22fd57415282-metrics-certs\") pod \"router-default-56768b9b-6x2s5\" (UID: \"6714944a-56b5-4b88-b10e-22fd57415282\") " pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:40.754628 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.754601 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:40.779517 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.779493 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:40.781701 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.781681 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/27bf791f-9d99-481e-8b1b-bc504cbeea9d-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gqhb2\" (UID: \"27bf791f-9d99-481e-8b1b-bc504cbeea9d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:40.848646 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.848617 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" Jul 13 17:21:40.879125 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.879103 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56768b9b-6x2s5"] Jul 13 17:21:40.881798 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:40.881765 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6714944a_56b5_4b88_b10e_22fd57415282.slice/crio-b2226711e4d083af93cad9122cd3844fa38b70ae6246bc07ef4c7041578b1587 WatchSource:0}: Error finding container b2226711e4d083af93cad9122cd3844fa38b70ae6246bc07ef4c7041578b1587: Status 404 returned error can't find the container with id b2226711e4d083af93cad9122cd3844fa38b70ae6246bc07ef4c7041578b1587 Jul 13 17:21:40.971388 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:40.971356 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2"] Jul 13 17:21:41.318109 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:41.318032 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" event={"ID":"27bf791f-9d99-481e-8b1b-bc504cbeea9d","Type":"ContainerStarted","Data":"5db4899fdbad91f66b31dcbba6eb160660324a1c054e17283962e7f4107b13e3"} Jul 13 17:21:41.319495 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:41.319471 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56768b9b-6x2s5" event={"ID":"6714944a-56b5-4b88-b10e-22fd57415282","Type":"ContainerStarted","Data":"de6b98eb8312b51b30d25aa71e93fdad8a9b9448f8bc27ae294a42914825a7b7"} Jul 13 17:21:41.319596 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:41.319510 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56768b9b-6x2s5" event={"ID":"6714944a-56b5-4b88-b10e-22fd57415282","Type":"ContainerStarted","Data":"b2226711e4d083af93cad9122cd3844fa38b70ae6246bc07ef4c7041578b1587"} Jul 13 17:21:41.339272 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:41.339232 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-56768b9b-6x2s5" podStartSLOduration=17.339217468 podStartE2EDuration="17.339217468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:21:41.338524454 +0000 UTC m=+239.495011340" watchObservedRunningTime="2026-07-13 17:21:41.339217468 +0000 UTC m=+239.495704373" Jul 13 17:21:41.755356 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:41.755313 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:41.758091 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:41.758068 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:42.325653 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:42.325625 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:42.326985 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:42.326963 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-56768b9b-6x2s5" Jul 13 17:21:43.329994 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:43.329960 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" event={"ID":"27bf791f-9d99-481e-8b1b-bc504cbeea9d","Type":"ContainerStarted","Data":"ee07f09850ac9d92867c7238fa66320b42267663968e52f4a6ed12146419e17b"} Jul 13 17:21:43.330372 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:43.330058 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" event={"ID":"27bf791f-9d99-481e-8b1b-bc504cbeea9d","Type":"ContainerStarted","Data":"378c14b4d43e7a9077a5027fe5b2ad9d96925e5366301cdc1bc2e09bb5355ef9"} Jul 13 17:21:43.350439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:43.350397 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gqhb2" podStartSLOduration=17.55676396 podStartE2EDuration="19.350382392s" totalImagesPullingTime="1.793618432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:21:43.349880993 +0000 UTC m=+241.506367869" watchObservedRunningTime="2026-07-13 17:21:43.350382392 +0000 UTC m=+241.506869268" Jul 13 17:21:44.711966 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:44.711937 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:44.712340 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:44.712002 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:44.712474 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:44.712456 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4caee1b8-5d63-4814-b089-967a32b2e038-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:44.714251 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:44.714231 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4caee1b8-5d63-4814-b089-967a32b2e038-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-289jk\" (UID: \"4caee1b8-5d63-4814-b089-967a32b2e038\") " pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:44.792536 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:44.792504 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m4nm5\"" Jul 13 17:21:44.801260 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:44.801239 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-289jk" Jul 13 17:21:44.937388 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:44.937344 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-289jk"] Jul 13 17:21:44.940080 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:44.940051 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4caee1b8_5d63_4814_b089_967a32b2e038.slice/crio-3d203ea3ce6b2ba042208b917f006dc326b132d57a1897233bb4a880e25de840 WatchSource:0}: Error finding container 3d203ea3ce6b2ba042208b917f006dc326b132d57a1897233bb4a880e25de840: Status 404 returned error can't find the container with id 3d203ea3ce6b2ba042208b917f006dc326b132d57a1897233bb4a880e25de840 Jul 13 17:21:45.336906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:45.336876 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-289jk" event={"ID":"4caee1b8-5d63-4814-b089-967a32b2e038","Type":"ContainerStarted","Data":"3d203ea3ce6b2ba042208b917f006dc326b132d57a1897233bb4a880e25de840"} Jul 13 17:21:49.355040 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:49.355007 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-289jk" event={"ID":"4caee1b8-5d63-4814-b089-967a32b2e038","Type":"ContainerStarted","Data":"3b2665c4ad1ff81a9f939d71dcbb1f3516ebb3b7a6f5b472a43ff208f6cd5b36"} Jul 13 17:21:50.359928 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:50.359887 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-289jk" event={"ID":"4caee1b8-5d63-4814-b089-967a32b2e038","Type":"ContainerStarted","Data":"3fc4b67b3062f23013df7745d44c1197b66638585d335a4981bee97da4b2f882"} Jul 13 17:21:52.367998 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:52.367963 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-289jk" event={"ID":"4caee1b8-5d63-4814-b089-967a32b2e038","Type":"ContainerStarted","Data":"3c9f9dcb5906ff24f2dc0a46635329dc87d5a8e81bae00de0139e10c1f712736"} Jul 13 17:21:52.386674 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:52.386586 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-289jk" podStartSLOduration=17.30825004 podStartE2EDuration="24.386570271s" totalImagesPullingTime="7.078320231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:21:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:21:52.385484365 +0000 UTC m=+250.541971241" watchObservedRunningTime="2026-07-13 17:21:52.386570271 +0000 UTC m=+250.543057145" Jul 13 17:21:54.080715 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:54.080661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:21:54.083033 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:54.083010 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87c1ac36-6870-4e28-97a9-1df7abf70a5e-metrics-certs\") pod \"network-metrics-daemon-48dkb\" (UID: \"87c1ac36-6870-4e28-97a9-1df7abf70a5e\") " pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:21:54.259871 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:54.259841 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gkqtt\"" Jul 13 17:21:54.267791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:54.267769 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-48dkb" Jul 13 17:21:54.388704 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:54.388677 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-48dkb"] Jul 13 17:21:54.392531 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:54.392505 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c1ac36_6870_4e28_97a9_1df7abf70a5e.slice/crio-807f069fb6dc283d20c4883e3f6cc2a207fad63658ad25ce35aefee76dbec504 WatchSource:0}: Error finding container 807f069fb6dc283d20c4883e3f6cc2a207fad63658ad25ce35aefee76dbec504: Status 404 returned error can't find the container with id 807f069fb6dc283d20c4883e3f6cc2a207fad63658ad25ce35aefee76dbec504 Jul 13 17:21:55.378952 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.378918 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-48dkb" event={"ID":"87c1ac36-6870-4e28-97a9-1df7abf70a5e","Type":"ContainerStarted","Data":"abea694bc2427524d5701df96cb56d50047e1aa2cd119e97ab9dc23c9c0894b3"} Jul 13 17:21:55.379214 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.378970 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-48dkb" event={"ID":"87c1ac36-6870-4e28-97a9-1df7abf70a5e","Type":"ContainerStarted","Data":"807f069fb6dc283d20c4883e3f6cc2a207fad63658ad25ce35aefee76dbec504"} Jul 13 17:21:55.820431 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.820403 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-tw5tt"] Jul 13 17:21:55.823966 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.823946 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-tw5tt" Jul 13 17:21:55.828489 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.828466 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 17:21:55.828818 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.828799 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 17:21:55.829035 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.829021 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9j7qv\"" Jul 13 17:21:55.842299 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.842266 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-tw5tt"] Jul 13 17:21:55.994530 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:55.994494 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjrqx\" (UniqueName: \"kubernetes.io/projected/56e2f450-991e-43cc-bfdf-3078b3768f44-kube-api-access-cjrqx\") pod \"downloads-68c9cbb497-tw5tt\" (UID: \"56e2f450-991e-43cc-bfdf-3078b3768f44\") " pod="openshift-console/downloads-68c9cbb497-tw5tt" Jul 13 17:21:56.095075 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.095008 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjrqx\" (UniqueName: \"kubernetes.io/projected/56e2f450-991e-43cc-bfdf-3078b3768f44-kube-api-access-cjrqx\") pod \"downloads-68c9cbb497-tw5tt\" (UID: \"56e2f450-991e-43cc-bfdf-3078b3768f44\") " pod="openshift-console/downloads-68c9cbb497-tw5tt" Jul 13 17:21:56.104448 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.104417 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjrqx\" (UniqueName: \"kubernetes.io/projected/56e2f450-991e-43cc-bfdf-3078b3768f44-kube-api-access-cjrqx\") pod \"downloads-68c9cbb497-tw5tt\" (UID: \"56e2f450-991e-43cc-bfdf-3078b3768f44\") " pod="openshift-console/downloads-68c9cbb497-tw5tt" Jul 13 17:21:56.132724 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.132698 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-tw5tt" Jul 13 17:21:56.253011 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.252979 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-tw5tt"] Jul 13 17:21:56.258618 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:56.258576 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e2f450_991e_43cc_bfdf_3078b3768f44.slice/crio-05a7aea6edf460624ede8ae417cb64ebcb04e615dc710b342bcbba98e855347a WatchSource:0}: Error finding container 05a7aea6edf460624ede8ae417cb64ebcb04e615dc710b342bcbba98e855347a: Status 404 returned error can't find the container with id 05a7aea6edf460624ede8ae417cb64ebcb04e615dc710b342bcbba98e855347a Jul 13 17:21:56.383844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.383815 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-48dkb" event={"ID":"87c1ac36-6870-4e28-97a9-1df7abf70a5e","Type":"ContainerStarted","Data":"f9d4ab619a6e37742dd3099d1bfc073d130f41b274b30e773936e3be6464a9bc"} Jul 13 17:21:56.384992 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.384972 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-tw5tt" event={"ID":"56e2f450-991e-43cc-bfdf-3078b3768f44","Type":"ContainerStarted","Data":"05a7aea6edf460624ede8ae417cb64ebcb04e615dc710b342bcbba98e855347a"} Jul 13 17:21:56.399388 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.399342 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-48dkb" podStartSLOduration=252.571959139 podStartE2EDuration="4m13.399310735s" totalImagesPullingTime="827.351596ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:21:56.398885815 +0000 UTC m=+254.555372700" watchObservedRunningTime="2026-07-13 17:21:56.399310735 +0000 UTC m=+254.555797609" Jul 13 17:21:56.699589 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.699515 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:56.701776 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.701747 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc37ddc9-06d8-4427-b437-aff8b7169805-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6nfw2\" (UID: \"cc37ddc9-06d8-4427-b437-aff8b7169805\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:56.951923 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.951859 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pjqtp\"" Jul 13 17:21:56.960680 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:56.960658 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" Jul 13 17:21:57.122747 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.122717 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2"] Jul 13 17:21:57.126239 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:57.126211 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc37ddc9_06d8_4427_b437_aff8b7169805.slice/crio-2f45539bb2f05bf70fdc25a051c3233f414d170f52b24007078f9062b44d89c7 WatchSource:0}: Error finding container 2f45539bb2f05bf70fdc25a051c3233f414d170f52b24007078f9062b44d89c7: Status 404 returned error can't find the container with id 2f45539bb2f05bf70fdc25a051c3233f414d170f52b24007078f9062b44d89c7 Jul 13 17:21:57.389240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.389204 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" event={"ID":"cc37ddc9-06d8-4427-b437-aff8b7169805","Type":"ContainerStarted","Data":"2f45539bb2f05bf70fdc25a051c3233f414d170f52b24007078f9062b44d89c7"} Jul 13 17:21:57.718066 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.717982 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57d87b85dc-nclgw"] Jul 13 17:21:57.722685 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.722657 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.726241 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.726216 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 17:21:57.726241 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.726231 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 17:21:57.727289 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.727030 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 17:21:57.727289 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.727033 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 17:21:57.727289 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.727068 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 17:21:57.727289 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.727123 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5bpwm\"" Jul 13 17:21:57.740393 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.740367 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d87b85dc-nclgw"] Jul 13 17:21:57.807643 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.807606 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn26n\" (UniqueName: \"kubernetes.io/projected/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-kube-api-access-nn26n\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.807643 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.807657 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-config\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.808003 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.807740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-serving-cert\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.808003 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.807831 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-oauth-config\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.808003 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.807891 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-service-ca\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.808003 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.807922 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-oauth-serving-cert\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.909224 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.909170 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-serving-cert\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.909224 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.909233 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-oauth-config\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.909501 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.909375 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-service-ca\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.909501 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.909422 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-oauth-serving-cert\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.909501 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.909482 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn26n\" (UniqueName: \"kubernetes.io/projected/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-kube-api-access-nn26n\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.909651 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.909523 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-config\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.910237 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.910175 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-service-ca\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.910383 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.910271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-config\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.910383 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.910294 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-oauth-serving-cert\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.912335 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.912269 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-oauth-config\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.912442 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.912411 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-serving-cert\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:57.918386 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:57.918341 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn26n\" (UniqueName: \"kubernetes.io/projected/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-kube-api-access-nn26n\") pod \"console-57d87b85dc-nclgw\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:58.035298 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:58.035210 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:21:58.185854 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:58.185822 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57d87b85dc-nclgw"] Jul 13 17:21:58.189209 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:21:58.189181 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28f8cb7_4726_4cc3_970b_d3e9b2174c01.slice/crio-0567dbada55783e8e463e0b43d1df84725667ad5e06ef1d58367f647cbcf3333 WatchSource:0}: Error finding container 0567dbada55783e8e463e0b43d1df84725667ad5e06ef1d58367f647cbcf3333: Status 404 returned error can't find the container with id 0567dbada55783e8e463e0b43d1df84725667ad5e06ef1d58367f647cbcf3333 Jul 13 17:21:58.394822 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:58.394756 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d87b85dc-nclgw" event={"ID":"e28f8cb7-4726-4cc3-970b-d3e9b2174c01","Type":"ContainerStarted","Data":"0567dbada55783e8e463e0b43d1df84725667ad5e06ef1d58367f647cbcf3333"} Jul 13 17:21:59.400899 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:59.400851 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" event={"ID":"cc37ddc9-06d8-4427-b437-aff8b7169805","Type":"ContainerStarted","Data":"d97772e2d0ccdeda76c1f172ae788ae9a0ab83c1f23e1d339d7bf6cda9c78265"} Jul 13 17:21:59.436306 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:21:59.436239 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6nfw2" podStartSLOduration=33.439939744 podStartE2EDuration="35.43622263s" totalImagesPullingTime="1.996282886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:21:59.43609007 +0000 UTC m=+257.592576947" watchObservedRunningTime="2026-07-13 17:21:59.43622263 +0000 UTC m=+257.592709504" Jul 13 17:22:02.757209 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.757175 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jqrt7"] Jul 13 17:22:02.760674 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.760648 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.763928 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.763864 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 17:22:02.763928 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.763877 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mb4qs\"" Jul 13 17:22:02.763928 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.763887 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 17:22:02.764160 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.763881 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 17:22:02.770967 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.770945 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jqrt7"] Jul 13 17:22:02.855262 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.855178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kqsj\" (UniqueName: \"kubernetes.io/projected/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-kube-api-access-6kqsj\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.855262 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.855256 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.855496 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.855290 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.855496 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.855380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.956286 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.956247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.956458 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.956307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.956458 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.956384 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.956458 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:02.956412 2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 13 17:22:02.956630 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:02.956485 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-tls podName:f397236d-344d-4817-bfc7-f0c3c9f6b0d0 nodeName:}" failed. No retries permitted until 2026-07-13 17:22:03.456463335 +0000 UTC m=+261.612950194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-tls") pod "prometheus-operator-64785b6675-jqrt7" (UID: "f397236d-344d-4817-bfc7-f0c3c9f6b0d0") : secret "prometheus-operator-tls" not found Jul 13 17:22:02.956630 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.956417 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kqsj\" (UniqueName: \"kubernetes.io/projected/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-kube-api-access-6kqsj\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.957239 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.957196 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-metrics-client-ca\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.959071 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.959041 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:02.969273 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:02.969251 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kqsj\" (UniqueName: \"kubernetes.io/projected/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-kube-api-access-6kqsj\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:03.420420 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:03.420379 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d87b85dc-nclgw" event={"ID":"e28f8cb7-4726-4cc3-970b-d3e9b2174c01","Type":"ContainerStarted","Data":"579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58"} Jul 13 17:22:03.440209 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:03.440152 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57d87b85dc-nclgw" podStartSLOduration=2.058283255 podStartE2EDuration="6.440132337s" totalImagesPullingTime="4.381849082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:21:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:03.438755267 +0000 UTC m=+261.595242142" watchObservedRunningTime="2026-07-13 17:22:03.440132337 +0000 UTC m=+261.596619212" Jul 13 17:22:03.461430 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:03.461401 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:03.463964 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:03.463942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f397236d-344d-4817-bfc7-f0c3c9f6b0d0-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-jqrt7\" (UID: \"f397236d-344d-4817-bfc7-f0c3c9f6b0d0\") " pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:03.672372 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:03.672289 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" Jul 13 17:22:03.806894 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:03.806827 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-jqrt7"] Jul 13 17:22:03.809853 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:22:03.809808 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf397236d_344d_4817_bfc7_f0c3c9f6b0d0.slice/crio-8bc235fe1e989d5ec46e02ce48b19fb2e519bc537da27dcb1a98a7e0265da06b WatchSource:0}: Error finding container 8bc235fe1e989d5ec46e02ce48b19fb2e519bc537da27dcb1a98a7e0265da06b: Status 404 returned error can't find the container with id 8bc235fe1e989d5ec46e02ce48b19fb2e519bc537da27dcb1a98a7e0265da06b Jul 13 17:22:04.425025 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:04.424984 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" event={"ID":"f397236d-344d-4817-bfc7-f0c3c9f6b0d0","Type":"ContainerStarted","Data":"8bc235fe1e989d5ec46e02ce48b19fb2e519bc537da27dcb1a98a7e0265da06b"} Jul 13 17:22:05.433128 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:05.433081 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" event={"ID":"f397236d-344d-4817-bfc7-f0c3c9f6b0d0","Type":"ContainerStarted","Data":"eabd2b74155189d348a5aeb12b774eeba63d98fdf94022dd34dce0450a2c4e6a"} Jul 13 17:22:05.433618 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:05.433151 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" event={"ID":"f397236d-344d-4817-bfc7-f0c3c9f6b0d0","Type":"ContainerStarted","Data":"be807c1cdde98a8ca73c995b2dd87e79b1dbb2651ed0fdcb4b0663f106da26ef"} Jul 13 17:22:05.452883 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:05.452467 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-jqrt7" podStartSLOduration=2.071086736 podStartE2EDuration="3.45244408s" totalImagesPullingTime="1.381357344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:05.450426773 +0000 UTC m=+263.606913647" watchObservedRunningTime="2026-07-13 17:22:05.45244408 +0000 UTC m=+263.608930946" Jul 13 17:22:07.136574 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.136542 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk"] Jul 13 17:22:07.141242 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.141206 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk"] Jul 13 17:22:07.141422 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.141363 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.142367 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.142341 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-x7k6v"] Jul 13 17:22:07.143818 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.143782 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 17:22:07.144137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.144091 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f2tsh\"" Jul 13 17:22:07.144465 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.144295 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 17:22:07.145802 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.145779 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.148764 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.148626 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qcdzz\"" Jul 13 17:22:07.148764 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.148663 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 17:22:07.148764 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.148704 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 17:22:07.148764 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.148743 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 17:22:07.160305 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.160281 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-x7k6v"] Jul 13 17:22:07.161428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.161408 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2v56r"] Jul 13 17:22:07.165013 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.164992 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.167446 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.167389 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 17:22:07.167658 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.167635 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wlrkx\"" Jul 13 17:22:07.167897 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.167876 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 17:22:07.167988 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.167877 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 17:22:07.191522 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191483 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-sys\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.191522 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-textfile\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.191743 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191549 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.191743 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191619 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e2a4f47-4904-4460-9672-59da0994d1ed-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.191743 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191668 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-accelerators-collector-config\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.191743 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-tls\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.191743 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191727 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.191980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191756 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-wtmp\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.191980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191791 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.191980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191828 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae89211d-8087-4afe-9718-6dc001e42016-metrics-client-ca\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.191980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191856 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-root\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.191980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.191980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.191965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.192242 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.192000 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.192242 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.192039 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e2a4f47-4904-4460-9672-59da0994d1ed-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.192242 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.192069 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f8fz\" (UniqueName: \"kubernetes.io/projected/ae89211d-8087-4afe-9718-6dc001e42016-kube-api-access-4f8fz\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.192242 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.192106 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mbtc\" (UniqueName: \"kubernetes.io/projected/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-api-access-4mbtc\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.192242 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.192140 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73fc4818-1724-4006-b1b4-650ef9b5f41d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.192242 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.192169 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhrx7\" (UniqueName: \"kubernetes.io/projected/73fc4818-1724-4006-b1b4-650ef9b5f41d-kube-api-access-zhrx7\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.293263 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293263 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293256 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae89211d-8087-4afe-9718-6dc001e42016-metrics-client-ca\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293285 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-root\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.293538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.293538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293391 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.293538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293418 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e2a4f47-4904-4460-9672-59da0994d1ed-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.293538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293422 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-root\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293444 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f8fz\" (UniqueName: \"kubernetes.io/projected/ae89211d-8087-4afe-9718-6dc001e42016-kube-api-access-4f8fz\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293495 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mbtc\" (UniqueName: \"kubernetes.io/projected/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-api-access-4mbtc\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73fc4818-1724-4006-b1b4-650ef9b5f41d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhrx7\" (UniqueName: \"kubernetes.io/projected/73fc4818-1724-4006-b1b4-650ef9b5f41d-kube-api-access-zhrx7\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293600 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-sys\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293626 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-textfile\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293653 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293686 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e2a4f47-4904-4460-9672-59da0994d1ed-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293730 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-accelerators-collector-config\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293766 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-tls\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293792 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.293906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293820 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-wtmp\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.294428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293983 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-wtmp\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.294428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.293985 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae89211d-8087-4afe-9718-6dc001e42016-metrics-client-ca\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.294428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.294037 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae89211d-8087-4afe-9718-6dc001e42016-sys\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.294428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.294095 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.294428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.294260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-textfile\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.294428 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:07.294369 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 17:22:07.294428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.294413 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e2a4f47-4904-4460-9672-59da0994d1ed-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.294791 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:07.294422 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-tls podName:ae89211d-8087-4afe-9718-6dc001e42016 nodeName:}" failed. No retries permitted until 2026-07-13 17:22:07.794404811 +0000 UTC m=+265.950891675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-tls") pod "node-exporter-2v56r" (UID: "ae89211d-8087-4afe-9718-6dc001e42016") : secret "node-exporter-tls" not found Jul 13 17:22:07.294791 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.294769 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-accelerators-collector-config\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.294909 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:07.294859 2579 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 13 17:22:07.294909 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:07.294907 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-tls podName:73fc4818-1724-4006-b1b4-650ef9b5f41d nodeName:}" failed. No retries permitted until 2026-07-13 17:22:07.794889899 +0000 UTC m=+265.951376752 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-4q4kk" (UID: "73fc4818-1724-4006-b1b4-650ef9b5f41d") : secret "openshift-state-metrics-tls" not found Jul 13 17:22:07.295048 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.295022 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5e2a4f47-4904-4460-9672-59da0994d1ed-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.295105 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.295042 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73fc4818-1724-4006-b1b4-650ef9b5f41d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.296643 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.296614 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.296973 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.296947 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.297076 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.296978 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.297134 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.297083 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.303054 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.303031 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f8fz\" (UniqueName: \"kubernetes.io/projected/ae89211d-8087-4afe-9718-6dc001e42016-kube-api-access-4f8fz\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.303159 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.303125 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhrx7\" (UniqueName: \"kubernetes.io/projected/73fc4818-1724-4006-b1b4-650ef9b5f41d-kube-api-access-zhrx7\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.303288 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.303266 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mbtc\" (UniqueName: \"kubernetes.io/projected/5e2a4f47-4904-4460-9672-59da0994d1ed-kube-api-access-4mbtc\") pod \"kube-state-metrics-785778ff74-x7k6v\" (UID: \"5e2a4f47-4904-4460-9672-59da0994d1ed\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.460971 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.460897 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" Jul 13 17:22:07.797845 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.797750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-tls\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.797845 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.797828 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:07.800482 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.800454 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae89211d-8087-4afe-9718-6dc001e42016-node-exporter-tls\") pod \"node-exporter-2v56r\" (UID: \"ae89211d-8087-4afe-9718-6dc001e42016\") " pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:07.800614 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:07.800520 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73fc4818-1724-4006-b1b4-650ef9b5f41d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-4q4kk\" (UID: \"73fc4818-1724-4006-b1b4-650ef9b5f41d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:08.036150 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:08.036094 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:22:08.036308 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:08.036172 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:22:08.037782 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:08.037751 2579 patch_prober.go:28] interesting pod/console-57d87b85dc-nclgw container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" start-of-body= Jul 13 17:22:08.037902 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:08.037816 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-57d87b85dc-nclgw" podUID="e28f8cb7-4726-4cc3-970b-d3e9b2174c01" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" Jul 13 17:22:08.054995 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:08.054929 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" Jul 13 17:22:08.075990 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:08.075962 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2v56r" Jul 13 17:22:11.417384 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.417346 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-fff5f5d57-fl8pw"] Jul 13 17:22:11.421486 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.421456 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.425790 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.425766 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 13 17:22:11.426033 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.426010 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 13 17:22:11.426777 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.426639 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x5hvg\"" Jul 13 17:22:11.426777 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.426706 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 13 17:22:11.426777 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.426756 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 13 17:22:11.426967 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.426889 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dmf7e8ofqr3bp\"" Jul 13 17:22:11.433423 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.433404 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-fff5f5d57-fl8pw"] Jul 13 17:22:11.534494 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.534452 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-client-ca-bundle\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.534654 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.534517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dba3fa76-12cb-4cef-81d8-1598a4474c1b-metrics-server-audit-profiles\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.534654 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.534548 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dba3fa76-12cb-4cef-81d8-1598a4474c1b-audit-log\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.534654 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.534566 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwfxz\" (UniqueName: \"kubernetes.io/projected/dba3fa76-12cb-4cef-81d8-1598a4474c1b-kube-api-access-wwfxz\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.534820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.534702 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba3fa76-12cb-4cef-81d8-1598a4474c1b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.534820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.534745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-secret-metrics-server-client-certs\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.534820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.534798 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-secret-metrics-server-tls\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.635795 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.635758 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-secret-metrics-server-tls\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.635941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.635804 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-client-ca-bundle\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.635941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.635838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dba3fa76-12cb-4cef-81d8-1598a4474c1b-metrics-server-audit-profiles\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.635941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.635869 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dba3fa76-12cb-4cef-81d8-1598a4474c1b-audit-log\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.635941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.635894 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwfxz\" (UniqueName: \"kubernetes.io/projected/dba3fa76-12cb-4cef-81d8-1598a4474c1b-kube-api-access-wwfxz\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.636153 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.635960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba3fa76-12cb-4cef-81d8-1598a4474c1b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.636153 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.635992 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-secret-metrics-server-client-certs\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.636516 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.636463 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dba3fa76-12cb-4cef-81d8-1598a4474c1b-audit-log\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.636931 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.636909 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dba3fa76-12cb-4cef-81d8-1598a4474c1b-metrics-server-audit-profiles\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.637104 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.637044 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dba3fa76-12cb-4cef-81d8-1598a4474c1b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.638582 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.638560 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-client-ca-bundle\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.638669 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.638649 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-secret-metrics-server-tls\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.638819 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.638792 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dba3fa76-12cb-4cef-81d8-1598a4474c1b-secret-metrics-server-client-certs\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.646754 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.646712 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwfxz\" (UniqueName: \"kubernetes.io/projected/dba3fa76-12cb-4cef-81d8-1598a4474c1b-kube-api-access-wwfxz\") pod \"metrics-server-fff5f5d57-fl8pw\" (UID: \"dba3fa76-12cb-4cef-81d8-1598a4474c1b\") " pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.733510 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.733446 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:11.897701 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.897673 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf"] Jul 13 17:22:11.900992 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.900969 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:11.903277 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.903250 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n5ccw\"" Jul 13 17:22:11.903456 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.903439 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 13 17:22:11.909212 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.909187 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf"] Jul 13 17:22:11.937797 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.937775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shj56\" (UniqueName: \"kubernetes.io/projected/50d8ede4-66fb-44c4-9324-740e23226fec-kube-api-access-shj56\") pod \"monitoring-plugin-5f68df7f58-svsrf\" (UID: \"50d8ede4-66fb-44c4-9324-740e23226fec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:11.937883 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:11.937869 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/50d8ede4-66fb-44c4-9324-740e23226fec-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-svsrf\" (UID: \"50d8ede4-66fb-44c4-9324-740e23226fec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:12.039034 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.038944 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/50d8ede4-66fb-44c4-9324-740e23226fec-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-svsrf\" (UID: \"50d8ede4-66fb-44c4-9324-740e23226fec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:12.039034 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.039019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shj56\" (UniqueName: \"kubernetes.io/projected/50d8ede4-66fb-44c4-9324-740e23226fec-kube-api-access-shj56\") pod \"monitoring-plugin-5f68df7f58-svsrf\" (UID: \"50d8ede4-66fb-44c4-9324-740e23226fec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:12.039252 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:12.039103 2579 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 13 17:22:12.039252 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:12.039179 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50d8ede4-66fb-44c4-9324-740e23226fec-monitoring-plugin-cert podName:50d8ede4-66fb-44c4-9324-740e23226fec nodeName:}" failed. No retries permitted until 2026-07-13 17:22:12.539158181 +0000 UTC m=+270.695645033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/50d8ede4-66fb-44c4-9324-740e23226fec-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-svsrf" (UID: "50d8ede4-66fb-44c4-9324-740e23226fec") : secret "monitoring-plugin-cert" not found Jul 13 17:22:12.047659 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.047630 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shj56\" (UniqueName: \"kubernetes.io/projected/50d8ede4-66fb-44c4-9324-740e23226fec-kube-api-access-shj56\") pod \"monitoring-plugin-5f68df7f58-svsrf\" (UID: \"50d8ede4-66fb-44c4-9324-740e23226fec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:12.514746 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.514714 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d87b85dc-nclgw"] Jul 13 17:22:12.544628 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.544594 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/50d8ede4-66fb-44c4-9324-740e23226fec-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-svsrf\" (UID: \"50d8ede4-66fb-44c4-9324-740e23226fec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:12.547018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.546998 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/50d8ede4-66fb-44c4-9324-740e23226fec-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-svsrf\" (UID: \"50d8ede4-66fb-44c4-9324-740e23226fec\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:12.553058 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.553032 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66b4f4557c-f9pmf"] Jul 13 17:22:12.556753 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.556734 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.569491 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.569462 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b4f4557c-f9pmf"] Jul 13 17:22:12.570033 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.570010 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 17:22:12.645969 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.645930 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-serving-cert\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.646117 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.646051 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-oauth-serving-cert\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.646117 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.646099 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-879wj\" (UniqueName: \"kubernetes.io/projected/585591f0-465e-45af-b083-c0f87bb4a172-kube-api-access-879wj\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.646207 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.646135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-service-ca\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.646207 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.646169 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-oauth-config\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.646207 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.646187 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-console-config\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.646359 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.646263 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-trusted-ca-bundle\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.747299 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.747259 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-879wj\" (UniqueName: \"kubernetes.io/projected/585591f0-465e-45af-b083-c0f87bb4a172-kube-api-access-879wj\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.747510 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.747311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-service-ca\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.747510 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.747386 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-oauth-config\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.747714 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.747682 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-console-config\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.747808 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.747780 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-trusted-ca-bundle\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.747869 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.747832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-serving-cert\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.747957 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.747939 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-oauth-serving-cert\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.748165 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.748139 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-service-ca\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.748531 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.748477 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-console-config\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.748712 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.748662 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-oauth-serving-cert\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.748712 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.748675 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-trusted-ca-bundle\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.750416 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.750397 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-oauth-config\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.750763 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.750745 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-serving-cert\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.755406 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.755383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-879wj\" (UniqueName: \"kubernetes.io/projected/585591f0-465e-45af-b083-c0f87bb4a172-kube-api-access-879wj\") pod \"console-66b4f4557c-f9pmf\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:12.813169 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.813090 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:12.869939 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:12.869904 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:13.700578 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:22:13.700542 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae89211d_8087_4afe_9718_6dc001e42016.slice/crio-1dc975f49feeb98efb18a6998f6b03d6be98ee128e620b85e0c9acd1d17b00fe WatchSource:0}: Error finding container 1dc975f49feeb98efb18a6998f6b03d6be98ee128e620b85e0c9acd1d17b00fe: Status 404 returned error can't find the container with id 1dc975f49feeb98efb18a6998f6b03d6be98ee128e620b85e0c9acd1d17b00fe Jul 13 17:22:13.932279 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:13.932248 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b4f4557c-f9pmf"] Jul 13 17:22:13.935855 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:22:13.935504 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585591f0_465e_45af_b083_c0f87bb4a172.slice/crio-7ada9e5047db698a898e508c560f9eb98a289a0eb73df88fedf4867cf31d03e3 WatchSource:0}: Error finding container 7ada9e5047db698a898e508c560f9eb98a289a0eb73df88fedf4867cf31d03e3: Status 404 returned error can't find the container with id 7ada9e5047db698a898e508c560f9eb98a289a0eb73df88fedf4867cf31d03e3 Jul 13 17:22:14.103825 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.103672 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk"] Jul 13 17:22:14.105144 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.105117 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-fff5f5d57-fl8pw"] Jul 13 17:22:14.106511 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:22:14.106483 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73fc4818_1724_4006_b1b4_650ef9b5f41d.slice/crio-9fab43770cb1286188df03d17d0db2ef9351f068d628fe60f78bdc40276e2aa2 WatchSource:0}: Error finding container 9fab43770cb1286188df03d17d0db2ef9351f068d628fe60f78bdc40276e2aa2: Status 404 returned error can't find the container with id 9fab43770cb1286188df03d17d0db2ef9351f068d628fe60f78bdc40276e2aa2 Jul 13 17:22:14.107210 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:22:14.107189 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba3fa76_12cb_4cef_81d8_1598a4474c1b.slice/crio-934c00d5170457b4cc6016edf145a045d17228512542cdd8f34b489b3461b4e2 WatchSource:0}: Error finding container 934c00d5170457b4cc6016edf145a045d17228512542cdd8f34b489b3461b4e2: Status 404 returned error can't find the container with id 934c00d5170457b4cc6016edf145a045d17228512542cdd8f34b489b3461b4e2 Jul 13 17:22:14.157704 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.157674 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf"] Jul 13 17:22:14.160909 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.160885 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-x7k6v"] Jul 13 17:22:14.165196 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:22:14.165161 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2a4f47_4904_4460_9672_59da0994d1ed.slice/crio-2d53c189b59ed65b5faef550942f1d038c5c34b6fc37ac1c4d81aca150747b6d WatchSource:0}: Error finding container 2d53c189b59ed65b5faef550942f1d038c5c34b6fc37ac1c4d81aca150747b6d: Status 404 returned error can't find the container with id 2d53c189b59ed65b5faef550942f1d038c5c34b6fc37ac1c4d81aca150747b6d Jul 13 17:22:14.473991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.473287 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-tw5tt" event={"ID":"56e2f450-991e-43cc-bfdf-3078b3768f44","Type":"ContainerStarted","Data":"1bf8dc36b5a15741879f1eea8b7812042fab175b453c11a3f05bccf0581a468b"} Jul 13 17:22:14.473991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.473666 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-tw5tt" Jul 13 17:22:14.481068 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.477804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" event={"ID":"73fc4818-1724-4006-b1b4-650ef9b5f41d","Type":"ContainerStarted","Data":"8a16f421261799ed018e619ef934c2367980655e6ed4adf3ab66580ae5b156b4"} Jul 13 17:22:14.481068 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.477906 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" event={"ID":"73fc4818-1724-4006-b1b4-650ef9b5f41d","Type":"ContainerStarted","Data":"a82136d16e40f11a1e63ebfac8bd3e7b6904508c99e05e015ef47b78368d93da"} Jul 13 17:22:14.481068 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.477943 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" event={"ID":"73fc4818-1724-4006-b1b4-650ef9b5f41d","Type":"ContainerStarted","Data":"9fab43770cb1286188df03d17d0db2ef9351f068d628fe60f78bdc40276e2aa2"} Jul 13 17:22:14.483228 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.483202 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b4f4557c-f9pmf" event={"ID":"585591f0-465e-45af-b083-c0f87bb4a172","Type":"ContainerStarted","Data":"53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0"} Jul 13 17:22:14.483354 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.483245 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b4f4557c-f9pmf" event={"ID":"585591f0-465e-45af-b083-c0f87bb4a172","Type":"ContainerStarted","Data":"7ada9e5047db698a898e508c560f9eb98a289a0eb73df88fedf4867cf31d03e3"} Jul 13 17:22:14.484980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.484949 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" event={"ID":"5e2a4f47-4904-4460-9672-59da0994d1ed","Type":"ContainerStarted","Data":"2d53c189b59ed65b5faef550942f1d038c5c34b6fc37ac1c4d81aca150747b6d"} Jul 13 17:22:14.486310 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.486277 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-tw5tt" Jul 13 17:22:14.486605 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.486576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" event={"ID":"50d8ede4-66fb-44c4-9324-740e23226fec","Type":"ContainerStarted","Data":"e26a4014a8c370332531f2d57536a9bd65c8c1e593da96eccaefbc8fe067e255"} Jul 13 17:22:14.488532 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.488507 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" event={"ID":"dba3fa76-12cb-4cef-81d8-1598a4474c1b","Type":"ContainerStarted","Data":"934c00d5170457b4cc6016edf145a045d17228512542cdd8f34b489b3461b4e2"} Jul 13 17:22:14.489966 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.489904 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2v56r" event={"ID":"ae89211d-8087-4afe-9718-6dc001e42016","Type":"ContainerStarted","Data":"1dc975f49feeb98efb18a6998f6b03d6be98ee128e620b85e0c9acd1d17b00fe"} Jul 13 17:22:14.508198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.508061 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-tw5tt" podStartSLOduration=1.925496954 podStartE2EDuration="19.508044635s" totalImagesPullingTime="17.582547681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:14.491906453 +0000 UTC m=+272.648393332" watchObservedRunningTime="2026-07-13 17:22:14.508044635 +0000 UTC m=+272.664531572" Jul 13 17:22:14.528901 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:14.527987 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66b4f4557c-f9pmf" podStartSLOduration=2.527969062 podStartE2EDuration="2.527969062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:22:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:14.527079511 +0000 UTC m=+272.683566388" watchObservedRunningTime="2026-07-13 17:22:14.527969062 +0000 UTC m=+272.684455971" Jul 13 17:22:15.501812 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:15.501609 2579 generic.go:356] "Generic (PLEG): container finished" podID="ae89211d-8087-4afe-9718-6dc001e42016" containerID="5dded1f3f6e8bdf19bfef977d050942b0edbaee7c6316101e48340e97481e974" exitCode=0 Jul 13 17:22:15.504241 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:15.503528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2v56r" event={"ID":"ae89211d-8087-4afe-9718-6dc001e42016","Type":"ContainerDied","Data":"5dded1f3f6e8bdf19bfef977d050942b0edbaee7c6316101e48340e97481e974"} Jul 13 17:22:18.524530 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.524493 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" event={"ID":"5e2a4f47-4904-4460-9672-59da0994d1ed","Type":"ContainerStarted","Data":"5dbb5c80a5ef415012de475cb10c6faac2c19276cb0af7f303e0c637ed815f4a"} Jul 13 17:22:18.525111 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.525088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" event={"ID":"5e2a4f47-4904-4460-9672-59da0994d1ed","Type":"ContainerStarted","Data":"422f97b8750e678cb0a9e89045a2a1bf533d5de29fad9c37acec6d429a0f8dc0"} Jul 13 17:22:18.526090 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.526066 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" event={"ID":"50d8ede4-66fb-44c4-9324-740e23226fec","Type":"ContainerStarted","Data":"fbf23d7c3fb20475d32b7afdce509abffb570d59736397dc48f53154a9b8670c"} Jul 13 17:22:18.526963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.526919 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:18.529980 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.529777 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" event={"ID":"dba3fa76-12cb-4cef-81d8-1598a4474c1b","Type":"ContainerStarted","Data":"755a38ca3e0ce4cbc2dd285332c96bb6d7fbd6ba5ad26880907b5e06f6803ff9"} Jul 13 17:22:18.532450 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.532358 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2v56r" event={"ID":"ae89211d-8087-4afe-9718-6dc001e42016","Type":"ContainerStarted","Data":"54938e8873a394c82e86a8c950b3eaaa95ab2e1de2a8d04e96a1acccc8959dae"} Jul 13 17:22:18.532450 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.532402 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2v56r" event={"ID":"ae89211d-8087-4afe-9718-6dc001e42016","Type":"ContainerStarted","Data":"93e998979023d6ad48ebd288d053befc73c7d007a9864a08bb3c224785412604"} Jul 13 17:22:18.536373 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.536304 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" Jul 13 17:22:18.537577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.537030 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" event={"ID":"73fc4818-1724-4006-b1b4-650ef9b5f41d","Type":"ContainerStarted","Data":"4f9087ed25df118ea0436ce7564d9b05552157401ab20435f5ce33b7b9733648"} Jul 13 17:22:18.561959 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.561780 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-svsrf" podStartSLOduration=3.641828292 podStartE2EDuration="7.561761608s" totalImagesPullingTime="3.919933316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:18.56086977 +0000 UTC m=+276.717356874" watchObservedRunningTime="2026-07-13 17:22:18.561761608 +0000 UTC m=+276.718248481" Jul 13 17:22:18.583223 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.583154 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2v56r" podStartSLOduration=10.200447476 podStartE2EDuration="11.583129851s" totalImagesPullingTime="1.266952997s" totalInitContainerRuntime="115.729378ms" isStatefulPod=true podCreationTimestamp="2026-07-13 17:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:18.581938913 +0000 UTC m=+276.738425789" watchObservedRunningTime="2026-07-13 17:22:18.583129851 +0000 UTC m=+276.739616727" Jul 13 17:22:18.601707 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.601637 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-4q4kk" podStartSLOduration=7.775647764 podStartE2EDuration="11.601612665s" totalImagesPullingTime="3.825964901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:18.600626222 +0000 UTC m=+276.757113111" watchObservedRunningTime="2026-07-13 17:22:18.601612665 +0000 UTC m=+276.758099543" Jul 13 17:22:18.641845 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:18.641743 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" podStartSLOduration=3.660217591 podStartE2EDuration="7.641707637s" totalImagesPullingTime="3.981490046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:18.620909169 +0000 UTC m=+276.777396044" watchObservedRunningTime="2026-07-13 17:22:18.641707637 +0000 UTC m=+276.798194516" Jul 13 17:22:19.544577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:19.544528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" event={"ID":"5e2a4f47-4904-4460-9672-59da0994d1ed","Type":"ContainerStarted","Data":"89fc30922597f9a434a20c334be817efcb9cc3c8a2573bed215813311d3e1931"} Jul 13 17:22:19.564894 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:19.564838 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-x7k6v" podStartSLOduration=8.64816893 podStartE2EDuration="12.564822371s" totalImagesPullingTime="3.916653441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:19.562620615 +0000 UTC m=+277.719107491" watchObservedRunningTime="2026-07-13 17:22:19.564822371 +0000 UTC m=+277.721309245" Jul 13 17:22:22.870128 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:22.870093 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:22.870617 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:22.870164 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:22.871238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:22.871217 2579 patch_prober.go:28] interesting pod/console-66b4f4557c-f9pmf container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.22:8443/health\": dial tcp 10.133.0.22:8443: connect: connection refused" start-of-body= Jul 13 17:22:22.871301 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:22.871260 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-66b4f4557c-f9pmf" podUID="585591f0-465e-45af-b083-c0f87bb4a172" containerName="console" probeResult="failure" output="Get \"https://10.133.0.22:8443/health\": dial tcp 10.133.0.22:8443: connect: connection refused" Jul 13 17:22:24.122216 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:24.122176 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cdlrj" podUID="9baab0f5-91c6-4a40-89b4-241bd5585973" Jul 13 17:22:24.568575 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:24.568548 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cdlrj" Jul 13 17:22:27.181858 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:27.181806 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:22:27.184134 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:27.184109 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9baab0f5-91c6-4a40-89b4-241bd5585973-metrics-tls\") pod \"dns-default-cdlrj\" (UID: \"9baab0f5-91c6-4a40-89b4-241bd5585973\") " pod="openshift-dns/dns-default-cdlrj" Jul 13 17:22:27.271632 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:27.271590 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f7qlt\"" Jul 13 17:22:27.279428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:27.279407 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cdlrj" Jul 13 17:22:27.541773 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:27.541746 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cdlrj"] Jul 13 17:22:27.543878 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:22:27.543853 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9baab0f5_91c6_4a40_89b4_241bd5585973.slice/crio-743b4d9b2e05939d00bbcc61169251ae4fb6af20f84cf614eee038f78ee6f5b0 WatchSource:0}: Error finding container 743b4d9b2e05939d00bbcc61169251ae4fb6af20f84cf614eee038f78ee6f5b0: Status 404 returned error can't find the container with id 743b4d9b2e05939d00bbcc61169251ae4fb6af20f84cf614eee038f78ee6f5b0 Jul 13 17:22:27.581032 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:27.581006 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cdlrj" event={"ID":"9baab0f5-91c6-4a40-89b4-241bd5585973","Type":"ContainerStarted","Data":"743b4d9b2e05939d00bbcc61169251ae4fb6af20f84cf614eee038f78ee6f5b0"} Jul 13 17:22:30.595195 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:30.595159 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cdlrj" event={"ID":"9baab0f5-91c6-4a40-89b4-241bd5585973","Type":"ContainerStarted","Data":"89e73fb9575a0a266d3482face377af40cbbe0aab4ee32e593c6075db866f0ba"} Jul 13 17:22:30.595632 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:30.595211 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cdlrj" event={"ID":"9baab0f5-91c6-4a40-89b4-241bd5585973","Type":"ContainerStarted","Data":"d7b22947df2554a0d94ef9df60f81d63735c90416fdae4d4f2111367906925f4"} Jul 13 17:22:30.595632 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:30.595281 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cdlrj" Jul 13 17:22:30.613874 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:30.613824 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cdlrj" podStartSLOduration=251.448888093 podStartE2EDuration="4m13.613809295s" totalImagesPullingTime="2.164921202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:30.612362913 +0000 UTC m=+288.768849788" watchObservedRunningTime="2026-07-13 17:22:30.613809295 +0000 UTC m=+288.770296170" Jul 13 17:22:31.734398 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:31.734367 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:31.734757 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:31.734430 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:32.874548 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:32.874516 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:32.878423 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:32.878401 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:22:37.535864 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.535818 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57d87b85dc-nclgw" podUID="e28f8cb7-4726-4cc3-970b-d3e9b2174c01" containerName="console" containerID="cri-o://579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58" gracePeriod=15 Jul 13 17:22:37.818869 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.818837 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:22:37.970018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.969983 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-config\") pod \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " Jul 13 17:22:37.970189 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970041 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-oauth-config\") pod \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " Jul 13 17:22:37.970189 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970063 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-oauth-serving-cert\") pod \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " Jul 13 17:22:37.970189 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970123 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-service-ca\") pod \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " Jul 13 17:22:37.970189 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970154 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nn26n\" (UniqueName: \"kubernetes.io/projected/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-kube-api-access-nn26n\") pod \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " Jul 13 17:22:37.970432 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970303 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-serving-cert\") pod \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\" (UID: \"e28f8cb7-4726-4cc3-970b-d3e9b2174c01\") " Jul 13 17:22:37.970490 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970467 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-config" (OuterVolumeSpecName: "console-config") pod "e28f8cb7-4726-4cc3-970b-d3e9b2174c01" (UID: "e28f8cb7-4726-4cc3-970b-d3e9b2174c01"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:22:37.970585 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970560 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e28f8cb7-4726-4cc3-970b-d3e9b2174c01" (UID: "e28f8cb7-4726-4cc3-970b-d3e9b2174c01"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:22:37.970729 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970702 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-oauth-serving-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:22:37.970856 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970732 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-config\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:22:37.970856 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.970807 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-service-ca" (OuterVolumeSpecName: "service-ca") pod "e28f8cb7-4726-4cc3-970b-d3e9b2174c01" (UID: "e28f8cb7-4726-4cc3-970b-d3e9b2174c01"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:22:37.972659 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.972633 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e28f8cb7-4726-4cc3-970b-d3e9b2174c01" (UID: "e28f8cb7-4726-4cc3-970b-d3e9b2174c01"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:22:37.972792 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.972767 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e28f8cb7-4726-4cc3-970b-d3e9b2174c01" (UID: "e28f8cb7-4726-4cc3-970b-d3e9b2174c01"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:22:37.972854 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:37.972773 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-kube-api-access-nn26n" (OuterVolumeSpecName: "kube-api-access-nn26n") pod "e28f8cb7-4726-4cc3-970b-d3e9b2174c01" (UID: "e28f8cb7-4726-4cc3-970b-d3e9b2174c01"). InnerVolumeSpecName "kube-api-access-nn26n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:22:38.071668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.071584 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-oauth-config\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:22:38.071668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.071613 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-service-ca\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:22:38.071668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.071626 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nn26n\" (UniqueName: \"kubernetes.io/projected/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-kube-api-access-nn26n\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:22:38.071668 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.071638 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e28f8cb7-4726-4cc3-970b-d3e9b2174c01-console-serving-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:22:38.624834 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.624781 2579 generic.go:356] "Generic (PLEG): container finished" podID="e28f8cb7-4726-4cc3-970b-d3e9b2174c01" containerID="579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58" exitCode=0 Jul 13 17:22:38.625298 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.624841 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57d87b85dc-nclgw" Jul 13 17:22:38.625298 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.624871 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d87b85dc-nclgw" event={"ID":"e28f8cb7-4726-4cc3-970b-d3e9b2174c01","Type":"ContainerDied","Data":"579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58"} Jul 13 17:22:38.625298 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.624960 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57d87b85dc-nclgw" event={"ID":"e28f8cb7-4726-4cc3-970b-d3e9b2174c01","Type":"ContainerDied","Data":"0567dbada55783e8e463e0b43d1df84725667ad5e06ef1d58367f647cbcf3333"} Jul 13 17:22:38.625298 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.624995 2579 scope.go:117] "RemoveContainer" containerID="579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58" Jul 13 17:22:38.636911 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.636893 2579 scope.go:117] "RemoveContainer" containerID="579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58" Jul 13 17:22:38.637188 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:22:38.637168 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58\": container with ID starting with 579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58 not found: ID does not exist" containerID="579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58" Jul 13 17:22:38.637244 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.637196 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58"} err="failed to get container status \"579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58\": rpc error: code = NotFound desc = could not find container \"579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58\": container with ID starting with 579242695ef2937547ef59e808e0f411c993285725a5148ba6a0c3640fde7e58 not found: ID does not exist" Jul 13 17:22:38.646334 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.646296 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57d87b85dc-nclgw"] Jul 13 17:22:38.648777 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.648757 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57d87b85dc-nclgw"] Jul 13 17:22:38.660429 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:38.660408 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e28f8cb7-4726-4cc3-970b-d3e9b2174c01" path="/var/lib/kubelet/pods/e28f8cb7-4726-4cc3-970b-d3e9b2174c01/volumes" Jul 13 17:22:40.601345 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:40.601293 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cdlrj" Jul 13 17:22:42.458436 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:42.458410 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:22:42.458844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:42.458515 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:22:42.469856 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:42.469830 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 17:22:48.663434 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:48.663384 2579 generic.go:356] "Generic (PLEG): container finished" podID="7a9856f9-2976-466d-b07f-e43582a7d59a" containerID="91a483bdd6506901564aea205335a43a8a671e2035ce3b2054424037efb0acce" exitCode=0 Jul 13 17:22:48.663434 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:48.663430 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" event={"ID":"7a9856f9-2976-466d-b07f-e43582a7d59a","Type":"ContainerDied","Data":"91a483bdd6506901564aea205335a43a8a671e2035ce3b2054424037efb0acce"} Jul 13 17:22:48.663912 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:48.663776 2579 scope.go:117] "RemoveContainer" containerID="91a483bdd6506901564aea205335a43a8a671e2035ce3b2054424037efb0acce" Jul 13 17:22:49.669845 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:49.669802 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hql5g" event={"ID":"7a9856f9-2976-466d-b07f-e43582a7d59a","Type":"ContainerStarted","Data":"27907970822e73e446fb12064c173dca2098de22dc7791849b9d298dc27e3f2a"} Jul 13 17:22:51.738842 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:51.738815 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:22:51.742722 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:22:51.742698 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-fff5f5d57-fl8pw" Jul 13 17:23:35.656344 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.656285 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5798bdc447-zjs9c"] Jul 13 17:23:35.656875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.656631 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28f8cb7-4726-4cc3-970b-d3e9b2174c01" containerName="console" Jul 13 17:23:35.656875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.656643 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28f8cb7-4726-4cc3-970b-d3e9b2174c01" containerName="console" Jul 13 17:23:35.656875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.656699 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28f8cb7-4726-4cc3-970b-d3e9b2174c01" containerName="console" Jul 13 17:23:35.658559 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.658536 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.677920 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.677887 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5798bdc447-zjs9c"] Jul 13 17:23:35.736535 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.736497 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-294xr\" (UniqueName: \"kubernetes.io/projected/22304325-8f58-418b-be2a-3563c13e7960-kube-api-access-294xr\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.736535 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.736541 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-trusted-ca-bundle\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.736747 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.736558 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-serving-cert\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.736747 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.736655 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-oauth-config\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.736747 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.736714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-console-config\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.736747 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.736733 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-service-ca\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.736886 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.736750 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-oauth-serving-cert\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.837739 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.837705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-trusted-ca-bundle\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.837739 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.837742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-serving-cert\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.837960 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.837765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-oauth-config\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.837960 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.837801 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-console-config\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.837960 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.837821 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-service-ca\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.837960 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.837839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-oauth-serving-cert\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.837960 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.837869 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-294xr\" (UniqueName: \"kubernetes.io/projected/22304325-8f58-418b-be2a-3563c13e7960-kube-api-access-294xr\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.838688 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.838657 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-console-config\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.838688 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.838689 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-trusted-ca-bundle\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.838908 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.838770 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-oauth-serving-cert\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.838908 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.838793 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-service-ca\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.841056 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.841033 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-oauth-config\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.841231 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.841037 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-serving-cert\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.848338 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.848297 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-294xr\" (UniqueName: \"kubernetes.io/projected/22304325-8f58-418b-be2a-3563c13e7960-kube-api-access-294xr\") pod \"console-5798bdc447-zjs9c\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:35.969548 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:35.969456 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:36.097743 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:36.097709 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5798bdc447-zjs9c"] Jul 13 17:23:36.100946 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:23:36.100915 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22304325_8f58_418b_be2a_3563c13e7960.slice/crio-3da16b623fda881ce3329597f6caa3b160e95422fd7be05a2afb74d9ff51591c WatchSource:0}: Error finding container 3da16b623fda881ce3329597f6caa3b160e95422fd7be05a2afb74d9ff51591c: Status 404 returned error can't find the container with id 3da16b623fda881ce3329597f6caa3b160e95422fd7be05a2afb74d9ff51591c Jul 13 17:23:36.858526 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:36.858486 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5798bdc447-zjs9c" event={"ID":"22304325-8f58-418b-be2a-3563c13e7960","Type":"ContainerStarted","Data":"221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6"} Jul 13 17:23:36.858920 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:36.858538 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5798bdc447-zjs9c" event={"ID":"22304325-8f58-418b-be2a-3563c13e7960","Type":"ContainerStarted","Data":"3da16b623fda881ce3329597f6caa3b160e95422fd7be05a2afb74d9ff51591c"} Jul 13 17:23:36.877449 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:36.877406 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5798bdc447-zjs9c" podStartSLOduration=1.8773875169999998 podStartE2EDuration="1.877387517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:23:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:23:36.876789582 +0000 UTC m=+355.033276458" watchObservedRunningTime="2026-07-13 17:23:36.877387517 +0000 UTC m=+355.033874393" Jul 13 17:23:45.970028 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:45.969973 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:45.970028 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:45.970049 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:45.974784 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:45.974757 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:46.901057 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:46.901029 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:23:46.949469 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:46.949425 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b4f4557c-f9pmf"] Jul 13 17:23:47.036213 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.036175 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b7dbbd65f-zfqb2"] Jul 13 17:23:47.038572 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.038544 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.050564 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.050540 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b7dbbd65f-zfqb2"] Jul 13 17:23:47.122951 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.122921 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq5t7\" (UniqueName: \"kubernetes.io/projected/11ba2824-84f5-46d4-a602-3b1d349639b9-kube-api-access-lq5t7\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.123072 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.122958 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-trusted-ca-bundle\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.123072 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.122988 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-serving-cert\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.123072 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.123053 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-oauth-config\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.123187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.123086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-service-ca\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.123187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.123109 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-console-config\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.123251 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.123191 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-oauth-serving-cert\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.223696 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.223629 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-oauth-config\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.223696 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.223661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-service-ca\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.223696 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.223679 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-console-config\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.223920 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.223712 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-oauth-serving-cert\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.223920 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.223732 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq5t7\" (UniqueName: \"kubernetes.io/projected/11ba2824-84f5-46d4-a602-3b1d349639b9-kube-api-access-lq5t7\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.223920 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.223753 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-trusted-ca-bundle\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.223920 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.223778 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-serving-cert\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.224500 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.224477 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-oauth-serving-cert\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.224612 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.224498 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-console-config\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.224612 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.224480 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-service-ca\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.224723 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.224694 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-trusted-ca-bundle\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.226107 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.226079 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-serving-cert\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.226203 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.226122 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-oauth-config\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.239450 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.239431 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq5t7\" (UniqueName: \"kubernetes.io/projected/11ba2824-84f5-46d4-a602-3b1d349639b9-kube-api-access-lq5t7\") pod \"console-b7dbbd65f-zfqb2\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.348151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.348116 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:47.474396 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.474347 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b7dbbd65f-zfqb2"] Jul 13 17:23:47.476513 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:23:47.476485 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ba2824_84f5_46d4_a602_3b1d349639b9.slice/crio-f666a2077782c7aa4026f5f4a1e9ba2ba1fc23988e0afdef3ff207f743da3939 WatchSource:0}: Error finding container f666a2077782c7aa4026f5f4a1e9ba2ba1fc23988e0afdef3ff207f743da3939: Status 404 returned error can't find the container with id f666a2077782c7aa4026f5f4a1e9ba2ba1fc23988e0afdef3ff207f743da3939 Jul 13 17:23:47.902411 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.902370 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7dbbd65f-zfqb2" event={"ID":"11ba2824-84f5-46d4-a602-3b1d349639b9","Type":"ContainerStarted","Data":"42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246"} Jul 13 17:23:47.902599 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.902435 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7dbbd65f-zfqb2" event={"ID":"11ba2824-84f5-46d4-a602-3b1d349639b9","Type":"ContainerStarted","Data":"f666a2077782c7aa4026f5f4a1e9ba2ba1fc23988e0afdef3ff207f743da3939"} Jul 13 17:23:47.922160 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:47.922114 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b7dbbd65f-zfqb2" podStartSLOduration=0.92210144 podStartE2EDuration="922.10144ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:23:47.920867438 +0000 UTC m=+366.077354313" watchObservedRunningTime="2026-07-13 17:23:47.92210144 +0000 UTC m=+366.078588315" Jul 13 17:23:57.348841 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:57.348797 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:57.349208 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:57.348864 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:57.353140 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:57.353116 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:57.944044 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:57.944017 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:23:57.997161 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:23:57.997123 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5798bdc447-zjs9c"] Jul 13 17:24:11.971775 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:11.971740 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66b4f4557c-f9pmf" podUID="585591f0-465e-45af-b083-c0f87bb4a172" containerName="console" containerID="cri-o://53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0" gracePeriod=15 Jul 13 17:24:12.209093 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.209070 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:24:12.309187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309124 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-console-config\") pod \"585591f0-465e-45af-b083-c0f87bb4a172\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " Jul 13 17:24:12.309187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309162 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-oauth-serving-cert\") pod \"585591f0-465e-45af-b083-c0f87bb4a172\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " Jul 13 17:24:12.309187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309181 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-oauth-config\") pod \"585591f0-465e-45af-b083-c0f87bb4a172\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " Jul 13 17:24:12.309433 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309202 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-879wj\" (UniqueName: \"kubernetes.io/projected/585591f0-465e-45af-b083-c0f87bb4a172-kube-api-access-879wj\") pod \"585591f0-465e-45af-b083-c0f87bb4a172\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " Jul 13 17:24:12.309433 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309370 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-service-ca\") pod \"585591f0-465e-45af-b083-c0f87bb4a172\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " Jul 13 17:24:12.309433 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309403 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-trusted-ca-bundle\") pod \"585591f0-465e-45af-b083-c0f87bb4a172\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " Jul 13 17:24:12.309585 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309542 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-serving-cert\") pod \"585591f0-465e-45af-b083-c0f87bb4a172\" (UID: \"585591f0-465e-45af-b083-c0f87bb4a172\") " Jul 13 17:24:12.309660 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309636 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-console-config" (OuterVolumeSpecName: "console-config") pod "585591f0-465e-45af-b083-c0f87bb4a172" (UID: "585591f0-465e-45af-b083-c0f87bb4a172"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:24:12.309718 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309671 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "585591f0-465e-45af-b083-c0f87bb4a172" (UID: "585591f0-465e-45af-b083-c0f87bb4a172"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:24:12.309718 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309696 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-service-ca" (OuterVolumeSpecName: "service-ca") pod "585591f0-465e-45af-b083-c0f87bb4a172" (UID: "585591f0-465e-45af-b083-c0f87bb4a172"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:24:12.309795 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309756 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "585591f0-465e-45af-b083-c0f87bb4a172" (UID: "585591f0-465e-45af-b083-c0f87bb4a172"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:24:12.309847 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309830 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-console-config\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:12.309889 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309853 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-oauth-serving-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:12.309889 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309866 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-service-ca\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:12.309889 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.309878 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/585591f0-465e-45af-b083-c0f87bb4a172-trusted-ca-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:12.311551 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.311525 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "585591f0-465e-45af-b083-c0f87bb4a172" (UID: "585591f0-465e-45af-b083-c0f87bb4a172"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:24:12.311755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.311728 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "585591f0-465e-45af-b083-c0f87bb4a172" (UID: "585591f0-465e-45af-b083-c0f87bb4a172"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:24:12.311755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.311741 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585591f0-465e-45af-b083-c0f87bb4a172-kube-api-access-879wj" (OuterVolumeSpecName: "kube-api-access-879wj") pod "585591f0-465e-45af-b083-c0f87bb4a172" (UID: "585591f0-465e-45af-b083-c0f87bb4a172"). InnerVolumeSpecName "kube-api-access-879wj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:24:12.411162 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.411141 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-serving-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:12.411162 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.411160 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/585591f0-465e-45af-b083-c0f87bb4a172-console-oauth-config\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:12.411277 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:12.411168 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-879wj\" (UniqueName: \"kubernetes.io/projected/585591f0-465e-45af-b083-c0f87bb4a172-kube-api-access-879wj\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:13.001571 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.001489 2579 generic.go:356] "Generic (PLEG): container finished" podID="585591f0-465e-45af-b083-c0f87bb4a172" containerID="53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0" exitCode=0 Jul 13 17:24:13.001571 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.001533 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b4f4557c-f9pmf" Jul 13 17:24:13.001571 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.001542 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b4f4557c-f9pmf" event={"ID":"585591f0-465e-45af-b083-c0f87bb4a172","Type":"ContainerDied","Data":"53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0"} Jul 13 17:24:13.002014 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.001585 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b4f4557c-f9pmf" event={"ID":"585591f0-465e-45af-b083-c0f87bb4a172","Type":"ContainerDied","Data":"7ada9e5047db698a898e508c560f9eb98a289a0eb73df88fedf4867cf31d03e3"} Jul 13 17:24:13.002014 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.001605 2579 scope.go:117] "RemoveContainer" containerID="53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0" Jul 13 17:24:13.011119 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.011102 2579 scope.go:117] "RemoveContainer" containerID="53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0" Jul 13 17:24:13.011371 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:24:13.011354 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0\": container with ID starting with 53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0 not found: ID does not exist" containerID="53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0" Jul 13 17:24:13.011423 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.011380 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0"} err="failed to get container status \"53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0\": rpc error: code = NotFound desc = could not find container \"53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0\": container with ID starting with 53f6236e134aeb28b54b51f04766feaed141aee74069cf4cdcfea85c72c965f0 not found: ID does not exist" Jul 13 17:24:13.020387 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.020363 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b4f4557c-f9pmf"] Jul 13 17:24:13.024202 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:13.024170 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66b4f4557c-f9pmf"] Jul 13 17:24:14.660375 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:14.660346 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585591f0-465e-45af-b083-c0f87bb4a172" path="/var/lib/kubelet/pods/585591f0-465e-45af-b083-c0f87bb4a172/volumes" Jul 13 17:24:23.020394 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.020360 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5798bdc447-zjs9c" podUID="22304325-8f58-418b-be2a-3563c13e7960" containerName="console" containerID="cri-o://221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6" gracePeriod=15 Jul 13 17:24:23.250601 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.250567 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:24:23.292941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.292877 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-oauth-serving-cert\") pod \"22304325-8f58-418b-be2a-3563c13e7960\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " Jul 13 17:24:23.292941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.292915 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-294xr\" (UniqueName: \"kubernetes.io/projected/22304325-8f58-418b-be2a-3563c13e7960-kube-api-access-294xr\") pod \"22304325-8f58-418b-be2a-3563c13e7960\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " Jul 13 17:24:23.292941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.292939 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-console-config\") pod \"22304325-8f58-418b-be2a-3563c13e7960\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " Jul 13 17:24:23.293151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.292974 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-service-ca\") pod \"22304325-8f58-418b-be2a-3563c13e7960\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " Jul 13 17:24:23.293151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293086 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-serving-cert\") pod \"22304325-8f58-418b-be2a-3563c13e7960\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " Jul 13 17:24:23.293151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293138 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-oauth-config\") pod \"22304325-8f58-418b-be2a-3563c13e7960\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " Jul 13 17:24:23.293285 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293176 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-trusted-ca-bundle\") pod \"22304325-8f58-418b-be2a-3563c13e7960\" (UID: \"22304325-8f58-418b-be2a-3563c13e7960\") " Jul 13 17:24:23.293370 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293279 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "22304325-8f58-418b-be2a-3563c13e7960" (UID: "22304325-8f58-418b-be2a-3563c13e7960"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:24:23.293370 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293305 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-service-ca" (OuterVolumeSpecName: "service-ca") pod "22304325-8f58-418b-be2a-3563c13e7960" (UID: "22304325-8f58-418b-be2a-3563c13e7960"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:24:23.293564 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293431 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-console-config" (OuterVolumeSpecName: "console-config") pod "22304325-8f58-418b-be2a-3563c13e7960" (UID: "22304325-8f58-418b-be2a-3563c13e7960"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:24:23.293564 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293557 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-oauth-serving-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:23.293731 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293575 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-service-ca\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:23.293731 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.293624 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "22304325-8f58-418b-be2a-3563c13e7960" (UID: "22304325-8f58-418b-be2a-3563c13e7960"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:24:23.295045 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.295020 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22304325-8f58-418b-be2a-3563c13e7960-kube-api-access-294xr" (OuterVolumeSpecName: "kube-api-access-294xr") pod "22304325-8f58-418b-be2a-3563c13e7960" (UID: "22304325-8f58-418b-be2a-3563c13e7960"). InnerVolumeSpecName "kube-api-access-294xr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:24:23.295155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.295131 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "22304325-8f58-418b-be2a-3563c13e7960" (UID: "22304325-8f58-418b-be2a-3563c13e7960"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:24:23.295222 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.295203 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "22304325-8f58-418b-be2a-3563c13e7960" (UID: "22304325-8f58-418b-be2a-3563c13e7960"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:24:23.394615 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.394595 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-294xr\" (UniqueName: \"kubernetes.io/projected/22304325-8f58-418b-be2a-3563c13e7960-kube-api-access-294xr\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:23.394615 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.394615 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-console-config\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:23.394719 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.394625 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-serving-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:23.394719 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.394633 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22304325-8f58-418b-be2a-3563c13e7960-console-oauth-config\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:23.394719 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:23.394641 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22304325-8f58-418b-be2a-3563c13e7960-trusted-ca-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:24:24.045336 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.045290 2579 generic.go:356] "Generic (PLEG): container finished" podID="22304325-8f58-418b-be2a-3563c13e7960" containerID="221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6" exitCode=0 Jul 13 17:24:24.045702 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.045340 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5798bdc447-zjs9c" Jul 13 17:24:24.045702 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.045364 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5798bdc447-zjs9c" event={"ID":"22304325-8f58-418b-be2a-3563c13e7960","Type":"ContainerDied","Data":"221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6"} Jul 13 17:24:24.045702 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.045415 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5798bdc447-zjs9c" event={"ID":"22304325-8f58-418b-be2a-3563c13e7960","Type":"ContainerDied","Data":"3da16b623fda881ce3329597f6caa3b160e95422fd7be05a2afb74d9ff51591c"} Jul 13 17:24:24.045702 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.045446 2579 scope.go:117] "RemoveContainer" containerID="221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6" Jul 13 17:24:24.053510 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.053495 2579 scope.go:117] "RemoveContainer" containerID="221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6" Jul 13 17:24:24.053760 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:24:24.053746 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6\": container with ID starting with 221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6 not found: ID does not exist" containerID="221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6" Jul 13 17:24:24.053805 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.053766 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6"} err="failed to get container status \"221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6\": rpc error: code = NotFound desc = could not find container \"221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6\": container with ID starting with 221ecf898d7aebf2e6ecd8b3a459a761fc4fd28e3a48cf9471a9def41f8a95b6 not found: ID does not exist" Jul 13 17:24:24.066098 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.066078 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5798bdc447-zjs9c"] Jul 13 17:24:24.072263 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.072238 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5798bdc447-zjs9c"] Jul 13 17:24:24.659659 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:24:24.659636 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22304325-8f58-418b-be2a-3563c13e7960" path="/var/lib/kubelet/pods/22304325-8f58-418b-be2a-3563c13e7960/volumes" Jul 13 17:25:05.684380 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.684275 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t"] Jul 13 17:25:05.684820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.684615 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22304325-8f58-418b-be2a-3563c13e7960" containerName="console" Jul 13 17:25:05.684820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.684632 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="22304325-8f58-418b-be2a-3563c13e7960" containerName="console" Jul 13 17:25:05.684820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.684644 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585591f0-465e-45af-b083-c0f87bb4a172" containerName="console" Jul 13 17:25:05.684820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.684649 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="585591f0-465e-45af-b083-c0f87bb4a172" containerName="console" Jul 13 17:25:05.684820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.684707 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="585591f0-465e-45af-b083-c0f87bb4a172" containerName="console" Jul 13 17:25:05.684820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.684714 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="22304325-8f58-418b-be2a-3563c13e7960" containerName="console" Jul 13 17:25:05.687674 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.687657 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.690053 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.690030 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d4rbr\"" Jul 13 17:25:05.690481 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.690462 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 17:25:05.690801 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.690787 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 17:25:05.698160 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.698138 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t"] Jul 13 17:25:05.807181 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.807148 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.807372 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.807205 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9r2h\" (UniqueName: \"kubernetes.io/projected/489a3052-c1ec-4495-bb92-8dd23f6f87fc-kube-api-access-n9r2h\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.807372 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.807234 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.908058 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.908020 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.908215 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.908088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9r2h\" (UniqueName: \"kubernetes.io/projected/489a3052-c1ec-4495-bb92-8dd23f6f87fc-kube-api-access-n9r2h\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.908215 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.908116 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.908517 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.908495 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.908517 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.908510 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.917481 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.917462 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9r2h\" (UniqueName: \"kubernetes.io/projected/489a3052-c1ec-4495-bb92-8dd23f6f87fc-kube-api-access-n9r2h\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:05.997591 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:05.997501 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:06.127949 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:06.127915 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t"] Jul 13 17:25:06.130963 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:25:06.130939 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod489a3052_c1ec_4495_bb92_8dd23f6f87fc.slice/crio-d5da4238178b9ded0c05fcd5e5ba259ee76b5d4581fe8deffe0e4403ecc25acb WatchSource:0}: Error finding container d5da4238178b9ded0c05fcd5e5ba259ee76b5d4581fe8deffe0e4403ecc25acb: Status 404 returned error can't find the container with id d5da4238178b9ded0c05fcd5e5ba259ee76b5d4581fe8deffe0e4403ecc25acb Jul 13 17:25:06.132772 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:06.132753 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:25:06.210507 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:06.210471 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" event={"ID":"489a3052-c1ec-4495-bb92-8dd23f6f87fc","Type":"ContainerStarted","Data":"d5da4238178b9ded0c05fcd5e5ba259ee76b5d4581fe8deffe0e4403ecc25acb"} Jul 13 17:25:11.239121 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:11.239089 2579 generic.go:356] "Generic (PLEG): container finished" podID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerID="1f863f9190344f08306a4c8f5607262dd6a2951e93f763577644a2783f33c272" exitCode=0 Jul 13 17:25:11.239541 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:11.239170 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" event={"ID":"489a3052-c1ec-4495-bb92-8dd23f6f87fc","Type":"ContainerDied","Data":"1f863f9190344f08306a4c8f5607262dd6a2951e93f763577644a2783f33c272"} Jul 13 17:25:14.253159 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:14.253127 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" event={"ID":"489a3052-c1ec-4495-bb92-8dd23f6f87fc","Type":"ContainerStarted","Data":"c16b5dd4d596f75dc1668c672e58ef063cd529cc6d9f36ce3d364dce9b6838fb"} Jul 13 17:25:15.259070 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:15.259039 2579 generic.go:356] "Generic (PLEG): container finished" podID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerID="c16b5dd4d596f75dc1668c672e58ef063cd529cc6d9f36ce3d364dce9b6838fb" exitCode=0 Jul 13 17:25:15.259422 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:15.259088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" event={"ID":"489a3052-c1ec-4495-bb92-8dd23f6f87fc","Type":"ContainerDied","Data":"c16b5dd4d596f75dc1668c672e58ef063cd529cc6d9f36ce3d364dce9b6838fb"} Jul 13 17:25:23.295415 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:23.295385 2579 generic.go:356] "Generic (PLEG): container finished" podID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerID="d9422d8a310d8a5e65aaff2add1e07986d92072b9d12515737849885f59f9203" exitCode=0 Jul 13 17:25:23.295727 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:23.295459 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" event={"ID":"489a3052-c1ec-4495-bb92-8dd23f6f87fc","Type":"ContainerDied","Data":"d9422d8a310d8a5e65aaff2add1e07986d92072b9d12515737849885f59f9203"} Jul 13 17:25:24.466281 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.466256 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:24.548701 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.548674 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-util\") pod \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " Jul 13 17:25:24.548813 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.548708 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9r2h\" (UniqueName: \"kubernetes.io/projected/489a3052-c1ec-4495-bb92-8dd23f6f87fc-kube-api-access-n9r2h\") pod \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " Jul 13 17:25:24.548813 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.548735 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-bundle\") pod \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\" (UID: \"489a3052-c1ec-4495-bb92-8dd23f6f87fc\") " Jul 13 17:25:24.549340 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.549303 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-bundle" (OuterVolumeSpecName: "bundle") pod "489a3052-c1ec-4495-bb92-8dd23f6f87fc" (UID: "489a3052-c1ec-4495-bb92-8dd23f6f87fc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:25:24.550829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.550773 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/489a3052-c1ec-4495-bb92-8dd23f6f87fc-kube-api-access-n9r2h" (OuterVolumeSpecName: "kube-api-access-n9r2h") pod "489a3052-c1ec-4495-bb92-8dd23f6f87fc" (UID: "489a3052-c1ec-4495-bb92-8dd23f6f87fc"). InnerVolumeSpecName "kube-api-access-n9r2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:25:24.552561 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.552541 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-util" (OuterVolumeSpecName: "util") pod "489a3052-c1ec-4495-bb92-8dd23f6f87fc" (UID: "489a3052-c1ec-4495-bb92-8dd23f6f87fc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:25:24.649355 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.649313 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-util\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:24.649355 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.649349 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9r2h\" (UniqueName: \"kubernetes.io/projected/489a3052-c1ec-4495-bb92-8dd23f6f87fc-kube-api-access-n9r2h\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:24.649355 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:24.649359 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/489a3052-c1ec-4495-bb92-8dd23f6f87fc-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:25.304920 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:25.304897 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" Jul 13 17:25:25.305067 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:25.304917 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mzg7t" event={"ID":"489a3052-c1ec-4495-bb92-8dd23f6f87fc","Type":"ContainerDied","Data":"d5da4238178b9ded0c05fcd5e5ba259ee76b5d4581fe8deffe0e4403ecc25acb"} Jul 13 17:25:25.305067 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:25.304955 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5da4238178b9ded0c05fcd5e5ba259ee76b5d4581fe8deffe0e4403ecc25acb" Jul 13 17:25:29.846285 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.846246 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5"] Jul 13 17:25:29.846850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.846696 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerName="extract" Jul 13 17:25:29.846850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.846715 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerName="extract" Jul 13 17:25:29.846850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.846735 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerName="pull" Jul 13 17:25:29.846850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.846742 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerName="pull" Jul 13 17:25:29.846850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.846763 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerName="util" Jul 13 17:25:29.846850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.846772 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerName="util" Jul 13 17:25:29.846850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.846840 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="489a3052-c1ec-4495-bb92-8dd23f6f87fc" containerName="extract" Jul 13 17:25:29.848993 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.848972 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" Jul 13 17:25:29.869099 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.869077 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 17:25:29.869217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.869162 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-s8n9w\"" Jul 13 17:25:29.869284 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.869222 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 17:25:29.884417 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.884396 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfcrd\" (UniqueName: \"kubernetes.io/projected/19733640-f0b3-4993-8795-d0e82f3129c0-kube-api-access-vfcrd\") pod \"cert-manager-operator-controller-manager-55c444bf86-bt8n5\" (UID: \"19733640-f0b3-4993-8795-d0e82f3129c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" Jul 13 17:25:29.884508 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.884431 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19733640-f0b3-4993-8795-d0e82f3129c0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bt8n5\" (UID: \"19733640-f0b3-4993-8795-d0e82f3129c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" Jul 13 17:25:29.893131 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.893110 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5"] Jul 13 17:25:29.985020 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.984986 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfcrd\" (UniqueName: \"kubernetes.io/projected/19733640-f0b3-4993-8795-d0e82f3129c0-kube-api-access-vfcrd\") pod \"cert-manager-operator-controller-manager-55c444bf86-bt8n5\" (UID: \"19733640-f0b3-4993-8795-d0e82f3129c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" Jul 13 17:25:29.985161 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.985030 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19733640-f0b3-4993-8795-d0e82f3129c0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bt8n5\" (UID: \"19733640-f0b3-4993-8795-d0e82f3129c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" Jul 13 17:25:29.985459 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:29.985439 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19733640-f0b3-4993-8795-d0e82f3129c0-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bt8n5\" (UID: \"19733640-f0b3-4993-8795-d0e82f3129c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" Jul 13 17:25:30.010195 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:30.010167 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfcrd\" (UniqueName: \"kubernetes.io/projected/19733640-f0b3-4993-8795-d0e82f3129c0-kube-api-access-vfcrd\") pod \"cert-manager-operator-controller-manager-55c444bf86-bt8n5\" (UID: \"19733640-f0b3-4993-8795-d0e82f3129c0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" Jul 13 17:25:30.157219 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:30.157188 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" Jul 13 17:25:30.322783 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:30.322758 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5"] Jul 13 17:25:30.325523 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:25:30.325495 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19733640_f0b3_4993_8795_d0e82f3129c0.slice/crio-4731b4243944f71c58ee54016b7cc19245b9e5761b0ce78db30acf11e82d5eff WatchSource:0}: Error finding container 4731b4243944f71c58ee54016b7cc19245b9e5761b0ce78db30acf11e82d5eff: Status 404 returned error can't find the container with id 4731b4243944f71c58ee54016b7cc19245b9e5761b0ce78db30acf11e82d5eff Jul 13 17:25:31.328723 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:31.328693 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" event={"ID":"19733640-f0b3-4993-8795-d0e82f3129c0","Type":"ContainerStarted","Data":"4731b4243944f71c58ee54016b7cc19245b9e5761b0ce78db30acf11e82d5eff"} Jul 13 17:25:32.334061 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:32.333977 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" event={"ID":"19733640-f0b3-4993-8795-d0e82f3129c0","Type":"ContainerStarted","Data":"51d18ace199577469594fdea37bde531471a60fe7305f9a7b9c68bfab4be92f3"} Jul 13 17:25:32.363006 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:32.362958 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bt8n5" podStartSLOduration=1.695015353 podStartE2EDuration="3.362943324s" totalImagesPullingTime="1.667927971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:25:32.361591684 +0000 UTC m=+470.518078560" watchObservedRunningTime="2026-07-13 17:25:32.362943324 +0000 UTC m=+470.519430197" Jul 13 17:25:32.978212 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:32.978179 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9"] Jul 13 17:25:32.981719 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:32.981697 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:32.984581 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:32.984563 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 17:25:32.985371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:32.985351 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d4rbr\"" Jul 13 17:25:32.995739 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:32.995722 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 17:25:33.009802 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.009780 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9"] Jul 13 17:25:33.013311 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.013288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.013432 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.013349 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.013432 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.013417 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc58t\" (UniqueName: \"kubernetes.io/projected/c527936b-2c72-47fd-be82-d16d82572785-kube-api-access-gc58t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.114113 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.114093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.114220 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.114122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.114220 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.114156 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc58t\" (UniqueName: \"kubernetes.io/projected/c527936b-2c72-47fd-be82-d16d82572785-kube-api-access-gc58t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.114514 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.114492 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.114591 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.114515 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.143044 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.143024 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc58t\" (UniqueName: \"kubernetes.io/projected/c527936b-2c72-47fd-be82-d16d82572785-kube-api-access-gc58t\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.290718 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.290671 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:33.435810 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:33.435787 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9"] Jul 13 17:25:33.438197 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:25:33.438172 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc527936b_2c72_47fd_be82_d16d82572785.slice/crio-ae0698d653a2911a474de168c16f9d101a968b7dbb8e47a2b87432d5e5ae9e2a WatchSource:0}: Error finding container ae0698d653a2911a474de168c16f9d101a968b7dbb8e47a2b87432d5e5ae9e2a: Status 404 returned error can't find the container with id ae0698d653a2911a474de168c16f9d101a968b7dbb8e47a2b87432d5e5ae9e2a Jul 13 17:25:34.345540 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:34.345507 2579 generic.go:356] "Generic (PLEG): container finished" podID="c527936b-2c72-47fd-be82-d16d82572785" containerID="cad0471ce8a968e15ee661151c18f219a518584eb424f1b3d0faf8ac22a93082" exitCode=0 Jul 13 17:25:34.345720 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:34.345589 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" event={"ID":"c527936b-2c72-47fd-be82-d16d82572785","Type":"ContainerDied","Data":"cad0471ce8a968e15ee661151c18f219a518584eb424f1b3d0faf8ac22a93082"} Jul 13 17:25:34.345720 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:34.345643 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" event={"ID":"c527936b-2c72-47fd-be82-d16d82572785","Type":"ContainerStarted","Data":"ae0698d653a2911a474de168c16f9d101a968b7dbb8e47a2b87432d5e5ae9e2a"} Jul 13 17:25:36.373816 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.373783 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw"] Jul 13 17:25:36.383617 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.383587 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" Jul 13 17:25:36.386149 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.386124 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 17:25:36.386978 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.386955 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 17:25:36.387094 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.386958 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-l4228\"" Jul 13 17:25:36.398349 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.398309 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw"] Jul 13 17:25:36.444304 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.444274 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjjpm\" (UniqueName: \"kubernetes.io/projected/885b24f2-d49d-494b-abf4-9ca21b473081-kube-api-access-bjjpm\") pod \"cert-manager-cainjector-559cdd8c87-8f8mw\" (UID: \"885b24f2-d49d-494b-abf4-9ca21b473081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" Jul 13 17:25:36.444455 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.444344 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/885b24f2-d49d-494b-abf4-9ca21b473081-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8f8mw\" (UID: \"885b24f2-d49d-494b-abf4-9ca21b473081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" Jul 13 17:25:36.545262 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.545225 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/885b24f2-d49d-494b-abf4-9ca21b473081-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8f8mw\" (UID: \"885b24f2-d49d-494b-abf4-9ca21b473081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" Jul 13 17:25:36.545430 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.545374 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjjpm\" (UniqueName: \"kubernetes.io/projected/885b24f2-d49d-494b-abf4-9ca21b473081-kube-api-access-bjjpm\") pod \"cert-manager-cainjector-559cdd8c87-8f8mw\" (UID: \"885b24f2-d49d-494b-abf4-9ca21b473081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" Jul 13 17:25:36.558404 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.558377 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/885b24f2-d49d-494b-abf4-9ca21b473081-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8f8mw\" (UID: \"885b24f2-d49d-494b-abf4-9ca21b473081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" Jul 13 17:25:36.558849 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.558830 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjjpm\" (UniqueName: \"kubernetes.io/projected/885b24f2-d49d-494b-abf4-9ca21b473081-kube-api-access-bjjpm\") pod \"cert-manager-cainjector-559cdd8c87-8f8mw\" (UID: \"885b24f2-d49d-494b-abf4-9ca21b473081\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" Jul 13 17:25:36.701971 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.701942 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" Jul 13 17:25:36.828813 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:36.828787 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw"] Jul 13 17:25:36.831093 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:25:36.831065 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod885b24f2_d49d_494b_abf4_9ca21b473081.slice/crio-114e1625ee21b2da83406c5ada43c851557da2ce0f8b4bb5edc2e4de02097ed2 WatchSource:0}: Error finding container 114e1625ee21b2da83406c5ada43c851557da2ce0f8b4bb5edc2e4de02097ed2: Status 404 returned error can't find the container with id 114e1625ee21b2da83406c5ada43c851557da2ce0f8b4bb5edc2e4de02097ed2 Jul 13 17:25:37.360700 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:37.360668 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" event={"ID":"885b24f2-d49d-494b-abf4-9ca21b473081","Type":"ContainerStarted","Data":"114e1625ee21b2da83406c5ada43c851557da2ce0f8b4bb5edc2e4de02097ed2"} Jul 13 17:25:37.362470 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:37.362444 2579 generic.go:356] "Generic (PLEG): container finished" podID="c527936b-2c72-47fd-be82-d16d82572785" containerID="ea81ecc8d8b0828ae6cf9f4d47bed1b86169273f92be118da2e7e1022bf45882" exitCode=0 Jul 13 17:25:37.362607 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:37.362503 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" event={"ID":"c527936b-2c72-47fd-be82-d16d82572785","Type":"ContainerDied","Data":"ea81ecc8d8b0828ae6cf9f4d47bed1b86169273f92be118da2e7e1022bf45882"} Jul 13 17:25:38.369867 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:38.369821 2579 generic.go:356] "Generic (PLEG): container finished" podID="c527936b-2c72-47fd-be82-d16d82572785" containerID="4c13157d04de8e6beb17fb233891d37ab6bc7be728eb0b2b917bc0a9105c7ad8" exitCode=0 Jul 13 17:25:38.370341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:38.369925 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" event={"ID":"c527936b-2c72-47fd-be82-d16d82572785","Type":"ContainerDied","Data":"4c13157d04de8e6beb17fb233891d37ab6bc7be728eb0b2b917bc0a9105c7ad8"} Jul 13 17:25:39.496298 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.496275 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:39.572635 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.572608 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-bundle\") pod \"c527936b-2c72-47fd-be82-d16d82572785\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " Jul 13 17:25:39.572802 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.572643 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-util\") pod \"c527936b-2c72-47fd-be82-d16d82572785\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " Jul 13 17:25:39.572802 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.572666 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc58t\" (UniqueName: \"kubernetes.io/projected/c527936b-2c72-47fd-be82-d16d82572785-kube-api-access-gc58t\") pod \"c527936b-2c72-47fd-be82-d16d82572785\" (UID: \"c527936b-2c72-47fd-be82-d16d82572785\") " Jul 13 17:25:39.572985 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.572963 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-bundle" (OuterVolumeSpecName: "bundle") pod "c527936b-2c72-47fd-be82-d16d82572785" (UID: "c527936b-2c72-47fd-be82-d16d82572785"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:25:39.574579 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.574549 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c527936b-2c72-47fd-be82-d16d82572785-kube-api-access-gc58t" (OuterVolumeSpecName: "kube-api-access-gc58t") pod "c527936b-2c72-47fd-be82-d16d82572785" (UID: "c527936b-2c72-47fd-be82-d16d82572785"). InnerVolumeSpecName "kube-api-access-gc58t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:25:39.576755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.576730 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-util" (OuterVolumeSpecName: "util") pod "c527936b-2c72-47fd-be82-d16d82572785" (UID: "c527936b-2c72-47fd-be82-d16d82572785"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:25:39.674148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.674120 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:39.674148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.674149 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c527936b-2c72-47fd-be82-d16d82572785-util\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:39.674352 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:39.674162 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc58t\" (UniqueName: \"kubernetes.io/projected/c527936b-2c72-47fd-be82-d16d82572785-kube-api-access-gc58t\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:40.379305 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:40.379277 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" Jul 13 17:25:40.379516 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:40.379312 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxfsv9" event={"ID":"c527936b-2c72-47fd-be82-d16d82572785","Type":"ContainerDied","Data":"ae0698d653a2911a474de168c16f9d101a968b7dbb8e47a2b87432d5e5ae9e2a"} Jul 13 17:25:40.379516 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:40.379373 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae0698d653a2911a474de168c16f9d101a968b7dbb8e47a2b87432d5e5ae9e2a" Jul 13 17:25:40.380856 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:40.380832 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" event={"ID":"885b24f2-d49d-494b-abf4-9ca21b473081","Type":"ContainerStarted","Data":"bc3dc0b68b8e10a8df439f2fc7752431c69ad417bff7ab68f5cc936447dd2046"} Jul 13 17:25:40.397429 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:40.397382 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8f8mw" podStartSLOduration=1.784174352 podStartE2EDuration="4.397370429s" totalImagesPullingTime="2.613196077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:25:40.3969931 +0000 UTC m=+478.553479972" watchObservedRunningTime="2026-07-13 17:25:40.397370429 +0000 UTC m=+478.553857301" Jul 13 17:25:41.404304 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.404274 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bc7ds"] Jul 13 17:25:41.404661 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.404588 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c527936b-2c72-47fd-be82-d16d82572785" containerName="pull" Jul 13 17:25:41.404661 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.404600 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c527936b-2c72-47fd-be82-d16d82572785" containerName="pull" Jul 13 17:25:41.404661 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.404609 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c527936b-2c72-47fd-be82-d16d82572785" containerName="extract" Jul 13 17:25:41.404661 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.404614 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c527936b-2c72-47fd-be82-d16d82572785" containerName="extract" Jul 13 17:25:41.404661 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.404626 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c527936b-2c72-47fd-be82-d16d82572785" containerName="util" Jul 13 17:25:41.404661 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.404631 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c527936b-2c72-47fd-be82-d16d82572785" containerName="util" Jul 13 17:25:41.404882 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.404685 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c527936b-2c72-47fd-be82-d16d82572785" containerName="extract" Jul 13 17:25:41.407742 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.407722 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:41.413921 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.413759 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2dk69\"" Jul 13 17:25:41.421397 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.421374 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bc7ds"] Jul 13 17:25:41.486190 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.486161 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxs2x\" (UniqueName: \"kubernetes.io/projected/cce6b1f9-052d-4ca4-85e1-1319f7502707-kube-api-access-wxs2x\") pod \"cert-manager-webhook-6486ffd997-bc7ds\" (UID: \"cce6b1f9-052d-4ca4-85e1-1319f7502707\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:41.486341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.486297 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cce6b1f9-052d-4ca4-85e1-1319f7502707-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bc7ds\" (UID: \"cce6b1f9-052d-4ca4-85e1-1319f7502707\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:41.587423 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.587397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxs2x\" (UniqueName: \"kubernetes.io/projected/cce6b1f9-052d-4ca4-85e1-1319f7502707-kube-api-access-wxs2x\") pod \"cert-manager-webhook-6486ffd997-bc7ds\" (UID: \"cce6b1f9-052d-4ca4-85e1-1319f7502707\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:41.587530 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.587470 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cce6b1f9-052d-4ca4-85e1-1319f7502707-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bc7ds\" (UID: \"cce6b1f9-052d-4ca4-85e1-1319f7502707\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:41.597826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.597799 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cce6b1f9-052d-4ca4-85e1-1319f7502707-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-bc7ds\" (UID: \"cce6b1f9-052d-4ca4-85e1-1319f7502707\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:41.597935 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.597917 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxs2x\" (UniqueName: \"kubernetes.io/projected/cce6b1f9-052d-4ca4-85e1-1319f7502707-kube-api-access-wxs2x\") pod \"cert-manager-webhook-6486ffd997-bc7ds\" (UID: \"cce6b1f9-052d-4ca4-85e1-1319f7502707\") " pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:41.716865 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.716816 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:41.844994 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:41.844968 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-bc7ds"] Jul 13 17:25:41.847512 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:25:41.847487 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcce6b1f9_052d_4ca4_85e1_1319f7502707.slice/crio-36df146e4c7a92e461ed657d81b04496b31ef35b97bb0f89f1100801d5156946 WatchSource:0}: Error finding container 36df146e4c7a92e461ed657d81b04496b31ef35b97bb0f89f1100801d5156946: Status 404 returned error can't find the container with id 36df146e4c7a92e461ed657d81b04496b31ef35b97bb0f89f1100801d5156946 Jul 13 17:25:42.390947 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:42.390914 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" event={"ID":"cce6b1f9-052d-4ca4-85e1-1319f7502707","Type":"ContainerStarted","Data":"fd00bdffe8f2ebc489e263630eac13fb1ef8015ee7d3234461cd382dad629f6b"} Jul 13 17:25:42.391128 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:42.390959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" event={"ID":"cce6b1f9-052d-4ca4-85e1-1319f7502707","Type":"ContainerStarted","Data":"36df146e4c7a92e461ed657d81b04496b31ef35b97bb0f89f1100801d5156946"} Jul 13 17:25:42.391128 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:42.391029 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:42.411188 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:42.411149 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" podStartSLOduration=1.411136558 podStartE2EDuration="1.411136558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:25:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:25:42.410051593 +0000 UTC m=+480.566538469" watchObservedRunningTime="2026-07-13 17:25:42.411136558 +0000 UTC m=+480.567623427" Jul 13 17:25:48.396945 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:48.396915 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-bc7ds" Jul 13 17:25:52.565761 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.565723 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8"] Jul 13 17:25:52.621777 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.621743 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8"] Jul 13 17:25:52.621949 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.621848 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.625190 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.625165 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 17:25:52.625401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.625360 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-d4rbr\"" Jul 13 17:25:52.626066 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.626046 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 17:25:52.669915 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.669886 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnzkh\" (UniqueName: \"kubernetes.io/projected/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-kube-api-access-pnzkh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.670079 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.669922 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.670079 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.669980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.770787 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.770758 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnzkh\" (UniqueName: \"kubernetes.io/projected/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-kube-api-access-pnzkh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.770787 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.770790 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.770958 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.770816 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.771211 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.771189 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.771252 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.771194 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.780704 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.780677 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnzkh\" (UniqueName: \"kubernetes.io/projected/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-kube-api-access-pnzkh\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:52.931574 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:52.931545 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:53.082548 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:53.082524 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8"] Jul 13 17:25:53.084916 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:25:53.084890 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fd2b7f2_f6f1_4adf_9456_30dd6a885a49.slice/crio-ee12380b62175f5cb8d34873bd9674fcd71784d025c9e28fe543267ff329b70e WatchSource:0}: Error finding container ee12380b62175f5cb8d34873bd9674fcd71784d025c9e28fe543267ff329b70e: Status 404 returned error can't find the container with id ee12380b62175f5cb8d34873bd9674fcd71784d025c9e28fe543267ff329b70e Jul 13 17:25:53.439735 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:53.439645 2579 generic.go:356] "Generic (PLEG): container finished" podID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerID="3ee593d4d4e3543268b3e8a45591ee187a0678997c87980e5ffc78926a282481" exitCode=0 Jul 13 17:25:53.439888 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:53.439730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" event={"ID":"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49","Type":"ContainerDied","Data":"3ee593d4d4e3543268b3e8a45591ee187a0678997c87980e5ffc78926a282481"} Jul 13 17:25:53.439888 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:53.439785 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" event={"ID":"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49","Type":"ContainerStarted","Data":"ee12380b62175f5cb8d34873bd9674fcd71784d025c9e28fe543267ff329b70e"} Jul 13 17:25:54.448090 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:54.448061 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" event={"ID":"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49","Type":"ContainerStarted","Data":"5ad28e97cb51c56e9f181fb0f9d773abf8314f91268b2dfafc81a05b34c99162"} Jul 13 17:25:55.453910 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:55.453877 2579 generic.go:356] "Generic (PLEG): container finished" podID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerID="5ad28e97cb51c56e9f181fb0f9d773abf8314f91268b2dfafc81a05b34c99162" exitCode=0 Jul 13 17:25:55.453910 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:55.453911 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" event={"ID":"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49","Type":"ContainerDied","Data":"5ad28e97cb51c56e9f181fb0f9d773abf8314f91268b2dfafc81a05b34c99162"} Jul 13 17:25:56.464042 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:56.464006 2579 generic.go:356] "Generic (PLEG): container finished" podID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerID="7422183ff3577ce3122eca25b9212068dba13f6bc6b38984dc5a081f7a49474a" exitCode=0 Jul 13 17:25:56.464439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:56.464087 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" event={"ID":"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49","Type":"ContainerDied","Data":"7422183ff3577ce3122eca25b9212068dba13f6bc6b38984dc5a081f7a49474a"} Jul 13 17:25:57.590614 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.590588 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:57.709498 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.709463 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnzkh\" (UniqueName: \"kubernetes.io/projected/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-kube-api-access-pnzkh\") pod \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " Jul 13 17:25:57.709696 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.709537 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-bundle\") pod \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " Jul 13 17:25:57.709696 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.709555 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-util\") pod \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\" (UID: \"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49\") " Jul 13 17:25:57.710233 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.710202 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-bundle" (OuterVolumeSpecName: "bundle") pod "2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" (UID: "2fd2b7f2-f6f1-4adf-9456-30dd6a885a49"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:25:57.711515 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.711494 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-kube-api-access-pnzkh" (OuterVolumeSpecName: "kube-api-access-pnzkh") pod "2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" (UID: "2fd2b7f2-f6f1-4adf-9456-30dd6a885a49"). InnerVolumeSpecName "kube-api-access-pnzkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:25:57.717150 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.717106 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-util" (OuterVolumeSpecName: "util") pod "2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" (UID: "2fd2b7f2-f6f1-4adf-9456-30dd6a885a49"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:25:57.810445 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.810358 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnzkh\" (UniqueName: \"kubernetes.io/projected/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-kube-api-access-pnzkh\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:57.810445 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.810389 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:57.810445 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:57.810400 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fd2b7f2-f6f1-4adf-9456-30dd6a885a49-util\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:25:58.475486 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:58.475454 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" Jul 13 17:25:58.475673 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:58.475448 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affk2q8" event={"ID":"2fd2b7f2-f6f1-4adf-9456-30dd6a885a49","Type":"ContainerDied","Data":"ee12380b62175f5cb8d34873bd9674fcd71784d025c9e28fe543267ff329b70e"} Jul 13 17:25:58.475673 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:25:58.475589 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee12380b62175f5cb8d34873bd9674fcd71784d025c9e28fe543267ff329b70e" Jul 13 17:26:10.493596 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.493561 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx"] Jul 13 17:26:10.493978 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.493958 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerName="pull" Jul 13 17:26:10.494018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.493983 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerName="pull" Jul 13 17:26:10.494018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.493999 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerName="util" Jul 13 17:26:10.494018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.494004 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerName="util" Jul 13 17:26:10.494018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.494013 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerName="extract" Jul 13 17:26:10.494018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.494018 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerName="extract" Jul 13 17:26:10.494156 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.494089 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fd2b7f2-f6f1-4adf-9456-30dd6a885a49" containerName="extract" Jul 13 17:26:10.497372 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.497352 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.501721 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.501701 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 17:26:10.501837 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.501706 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 17:26:10.501947 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.501927 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 17:26:10.503561 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.503537 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 17:26:10.504233 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.504216 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2kjrn\"" Jul 13 17:26:10.538499 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.538465 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx"] Jul 13 17:26:10.614613 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.614574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62232eb5-da3c-4c6c-be28-50363cd0cf32-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.614613 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.614614 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/62232eb5-da3c-4c6c-be28-50363cd0cf32-manifests\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.614827 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.614656 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/62232eb5-da3c-4c6c-be28-50363cd0cf32-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.614827 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.614705 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4n7\" (UniqueName: \"kubernetes.io/projected/62232eb5-da3c-4c6c-be28-50363cd0cf32-kube-api-access-fb4n7\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.614827 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.614765 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62232eb5-da3c-4c6c-be28-50363cd0cf32-tmp\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.716083 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.716045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/62232eb5-da3c-4c6c-be28-50363cd0cf32-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.716267 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.716089 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4n7\" (UniqueName: \"kubernetes.io/projected/62232eb5-da3c-4c6c-be28-50363cd0cf32-kube-api-access-fb4n7\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.716267 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.716138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62232eb5-da3c-4c6c-be28-50363cd0cf32-tmp\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.716267 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.716189 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62232eb5-da3c-4c6c-be28-50363cd0cf32-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.716267 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.716218 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/62232eb5-da3c-4c6c-be28-50363cd0cf32-manifests\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.716671 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.716638 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/62232eb5-da3c-4c6c-be28-50363cd0cf32-manifests\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.718421 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.718399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62232eb5-da3c-4c6c-be28-50363cd0cf32-tmp\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.718539 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.718514 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/62232eb5-da3c-4c6c-be28-50363cd0cf32-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.718664 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.718645 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62232eb5-da3c-4c6c-be28-50363cd0cf32-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.727417 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.727391 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4n7\" (UniqueName: \"kubernetes.io/projected/62232eb5-da3c-4c6c-be28-50363cd0cf32-kube-api-access-fb4n7\") pod \"opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx\" (UID: \"62232eb5-da3c-4c6c-be28-50363cd0cf32\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.807582 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.807509 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:10.973180 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:10.973146 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx"] Jul 13 17:26:10.976866 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:26:10.976840 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62232eb5_da3c_4c6c_be28_50363cd0cf32.slice/crio-9936e27cd8f62230e32f7deaf17095b09391cc4c79da85b1d39d076156653cc6 WatchSource:0}: Error finding container 9936e27cd8f62230e32f7deaf17095b09391cc4c79da85b1d39d076156653cc6: Status 404 returned error can't find the container with id 9936e27cd8f62230e32f7deaf17095b09391cc4c79da85b1d39d076156653cc6 Jul 13 17:26:11.533638 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:11.533602 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" event={"ID":"62232eb5-da3c-4c6c-be28-50363cd0cf32","Type":"ContainerStarted","Data":"9936e27cd8f62230e32f7deaf17095b09391cc4c79da85b1d39d076156653cc6"} Jul 13 17:26:14.549967 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:14.549879 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" event={"ID":"62232eb5-da3c-4c6c-be28-50363cd0cf32","Type":"ContainerStarted","Data":"588d7bba82b6794d09aa661835d3b503195b4ea57e152a300c0ee1961ee25f7e"} Jul 13 17:26:15.560916 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:15.560880 2579 generic.go:356] "Generic (PLEG): container finished" podID="62232eb5-da3c-4c6c-be28-50363cd0cf32" containerID="588d7bba82b6794d09aa661835d3b503195b4ea57e152a300c0ee1961ee25f7e" exitCode=0 Jul 13 17:26:15.561374 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:15.560944 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" event={"ID":"62232eb5-da3c-4c6c-be28-50363cd0cf32","Type":"ContainerDied","Data":"588d7bba82b6794d09aa661835d3b503195b4ea57e152a300c0ee1961ee25f7e"} Jul 13 17:26:16.568215 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:16.568178 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" event={"ID":"62232eb5-da3c-4c6c-be28-50363cd0cf32","Type":"ContainerStarted","Data":"68ac28c1a34a9bbc230999dda525af8a4594c412b0fcc9057eacd4f0555b561a"} Jul 13 17:26:16.568568 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:16.568296 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:16.595459 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:16.595412 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" podStartSLOduration=1.943758646 podStartE2EDuration="6.595396764s" totalImagesPullingTime="3.571516809s" totalInitContainerRuntime="1.080121309s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:16.594555546 +0000 UTC m=+514.751042421" watchObservedRunningTime="2026-07-13 17:26:16.595396764 +0000 UTC m=+514.751883637" Jul 13 17:26:27.575143 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:27.575113 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx" Jul 13 17:26:31.217025 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.216997 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lk78q"] Jul 13 17:26:31.229797 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.229775 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:31.234788 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.234762 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 13 17:26:31.239398 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.239377 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lk78q"] Jul 13 17:26:31.367266 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.367239 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5zrp\" (UniqueName: \"kubernetes.io/projected/22e7f71a-9d76-4daf-9356-6822b3d39f6b-kube-api-access-s5zrp\") pod \"model-serving-api-54d89bd5fc-lk78q\" (UID: \"22e7f71a-9d76-4daf-9356-6822b3d39f6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:31.367399 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.367338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22e7f71a-9d76-4daf-9356-6822b3d39f6b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lk78q\" (UID: \"22e7f71a-9d76-4daf-9356-6822b3d39f6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:31.468280 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.468221 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5zrp\" (UniqueName: \"kubernetes.io/projected/22e7f71a-9d76-4daf-9356-6822b3d39f6b-kube-api-access-s5zrp\") pod \"model-serving-api-54d89bd5fc-lk78q\" (UID: \"22e7f71a-9d76-4daf-9356-6822b3d39f6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:31.468392 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.468290 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22e7f71a-9d76-4daf-9356-6822b3d39f6b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lk78q\" (UID: \"22e7f71a-9d76-4daf-9356-6822b3d39f6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:31.468432 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:26:31.468416 2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 13 17:26:31.468487 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:26:31.468476 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e7f71a-9d76-4daf-9356-6822b3d39f6b-tls-certs podName:22e7f71a-9d76-4daf-9356-6822b3d39f6b nodeName:}" failed. No retries permitted until 2026-07-13 17:26:31.968457758 +0000 UTC m=+530.124944611 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/22e7f71a-9d76-4daf-9356-6822b3d39f6b-tls-certs") pod "model-serving-api-54d89bd5fc-lk78q" (UID: "22e7f71a-9d76-4daf-9356-6822b3d39f6b") : secret "model-serving-api-tls" not found Jul 13 17:26:31.483041 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.483013 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5zrp\" (UniqueName: \"kubernetes.io/projected/22e7f71a-9d76-4daf-9356-6822b3d39f6b-kube-api-access-s5zrp\") pod \"model-serving-api-54d89bd5fc-lk78q\" (UID: \"22e7f71a-9d76-4daf-9356-6822b3d39f6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:31.920438 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.920408 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65c65495b5-64fdt"] Jul 13 17:26:31.923977 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.923959 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:31.928035 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.928013 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 13 17:26:31.928155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.928053 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 13 17:26:31.943396 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.943377 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ckm7x\"" Jul 13 17:26:31.952651 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.952632 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c65495b5-64fdt"] Jul 13 17:26:31.971355 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.971304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22e7f71a-9d76-4daf-9356-6822b3d39f6b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lk78q\" (UID: \"22e7f71a-9d76-4daf-9356-6822b3d39f6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:31.973482 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:31.973464 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22e7f71a-9d76-4daf-9356-6822b3d39f6b-tls-certs\") pod \"model-serving-api-54d89bd5fc-lk78q\" (UID: \"22e7f71a-9d76-4daf-9356-6822b3d39f6b\") " pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:32.072108 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.072086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e516dff-e9c7-4106-ae22-260904fd849d-tmp\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.072195 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.072120 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmds9\" (UniqueName: \"kubernetes.io/projected/1e516dff-e9c7-4106-ae22-260904fd849d-kube-api-access-lmds9\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.072195 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.072144 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e516dff-e9c7-4106-ae22-260904fd849d-tls-certs\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.139261 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.139243 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:32.173356 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.173285 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e516dff-e9c7-4106-ae22-260904fd849d-tls-certs\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.173456 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.173438 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e516dff-e9c7-4106-ae22-260904fd849d-tmp\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.173506 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.173491 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmds9\" (UniqueName: \"kubernetes.io/projected/1e516dff-e9c7-4106-ae22-260904fd849d-kube-api-access-lmds9\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.175485 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.175463 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e516dff-e9c7-4106-ae22-260904fd849d-tmp\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.175660 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.175639 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e516dff-e9c7-4106-ae22-260904fd849d-tls-certs\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.184745 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.184725 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmds9\" (UniqueName: \"kubernetes.io/projected/1e516dff-e9c7-4106-ae22-260904fd849d-kube-api-access-lmds9\") pod \"kube-auth-proxy-65c65495b5-64fdt\" (UID: \"1e516dff-e9c7-4106-ae22-260904fd849d\") " pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.233627 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.233599 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" Jul 13 17:26:32.273568 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.273532 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-lk78q"] Jul 13 17:26:32.275769 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:26:32.275741 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e7f71a_9d76_4daf_9356_6822b3d39f6b.slice/crio-2ddf98a049a3e467426e8bc4b3b087805ae9d1400a30a7a1a8b69b439ccb9e36 WatchSource:0}: Error finding container 2ddf98a049a3e467426e8bc4b3b087805ae9d1400a30a7a1a8b69b439ccb9e36: Status 404 returned error can't find the container with id 2ddf98a049a3e467426e8bc4b3b087805ae9d1400a30a7a1a8b69b439ccb9e36 Jul 13 17:26:32.365020 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.364995 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65c65495b5-64fdt"] Jul 13 17:26:32.366839 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:26:32.366810 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e516dff_e9c7_4106_ae22_260904fd849d.slice/crio-99e9bd2550e0a036ab1f2c5f9a59e7b51ad01ce5e1cb15330d658403491362a1 WatchSource:0}: Error finding container 99e9bd2550e0a036ab1f2c5f9a59e7b51ad01ce5e1cb15330d658403491362a1: Status 404 returned error can't find the container with id 99e9bd2550e0a036ab1f2c5f9a59e7b51ad01ce5e1cb15330d658403491362a1 Jul 13 17:26:32.638768 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.638724 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" event={"ID":"1e516dff-e9c7-4106-ae22-260904fd849d","Type":"ContainerStarted","Data":"99e9bd2550e0a036ab1f2c5f9a59e7b51ad01ce5e1cb15330d658403491362a1"} Jul 13 17:26:32.640117 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:32.640087 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" event={"ID":"22e7f71a-9d76-4daf-9356-6822b3d39f6b","Type":"ContainerStarted","Data":"2ddf98a049a3e467426e8bc4b3b087805ae9d1400a30a7a1a8b69b439ccb9e36"} Jul 13 17:26:35.658187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:35.658156 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" event={"ID":"22e7f71a-9d76-4daf-9356-6822b3d39f6b","Type":"ContainerStarted","Data":"c3bc09a5ad6f27974f9a9d622533352b9b220ca38f4c281f7f5d06c159cf045c"} Jul 13 17:26:35.658647 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:35.658309 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:35.659719 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:35.659697 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" event={"ID":"1e516dff-e9c7-4106-ae22-260904fd849d","Type":"ContainerStarted","Data":"1b9fc31ba22da869622f915bdaa3b66608fa09607ced9fc2c4f4094d8b0cdcbe"} Jul 13 17:26:35.690350 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:35.690284 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" podStartSLOduration=2.349941108 podStartE2EDuration="4.690270179s" totalImagesPullingTime="2.340329071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:35.688630889 +0000 UTC m=+533.845117769" watchObservedRunningTime="2026-07-13 17:26:35.690270179 +0000 UTC m=+533.846757052" Jul 13 17:26:35.719167 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:35.719113 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65c65495b5-64fdt" podStartSLOduration=1.740730251 podStartE2EDuration="4.719098274s" totalImagesPullingTime="2.978368023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:35.716705967 +0000 UTC m=+533.873192841" watchObservedRunningTime="2026-07-13 17:26:35.719098274 +0000 UTC m=+533.875585186" Jul 13 17:26:39.278836 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.278805 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj"] Jul 13 17:26:39.282212 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.282189 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.285223 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.285202 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 17:26:39.285736 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.285719 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 17:26:39.285793 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.285751 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jpn6p\"" Jul 13 17:26:39.285996 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.285980 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 17:26:39.286388 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.286370 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 17:26:39.310033 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.310004 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj"] Jul 13 17:26:39.326476 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.326452 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.326584 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.326503 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvktz\" (UniqueName: \"kubernetes.io/projected/9551e061-4d3f-4ede-9d70-50d88a56dc94-kube-api-access-xvktz\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.326584 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.326520 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.326700 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.326608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9551e061-4d3f-4ede-9d70-50d88a56dc94-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.326700 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.326640 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.326700 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.326671 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.326828 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.326749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.356652 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.356621 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7"] Jul 13 17:26:39.360569 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.360537 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.363600 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.363570 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xttrs\"" Jul 13 17:26:39.384733 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.384698 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7"] Jul 13 17:26:39.427686 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.427826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427701 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.427826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.427826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.427826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427807 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.428032 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427845 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95krk\" (UniqueName: \"kubernetes.io/projected/2af08a2d-6765-4d43-a546-9c11fbd32823-kube-api-access-95krk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.428032 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427885 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.428032 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af08a2d-6765-4d43-a546-9c11fbd32823-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.428032 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427951 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.428032 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.427976 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.428032 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.428021 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvktz\" (UniqueName: \"kubernetes.io/projected/9551e061-4d3f-4ede-9d70-50d88a56dc94-kube-api-access-xvktz\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.428345 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.428043 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.428345 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.428069 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.428345 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.428092 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.428345 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.428143 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.428345 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.428172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9551e061-4d3f-4ede-9d70-50d88a56dc94-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.428738 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.428709 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.430249 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.430206 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.430483 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.430463 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9551e061-4d3f-4ede-9d70-50d88a56dc94-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.430555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.430475 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.430848 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.430829 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9551e061-4d3f-4ede-9d70-50d88a56dc94-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.437683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.437663 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvktz\" (UniqueName: \"kubernetes.io/projected/9551e061-4d3f-4ede-9d70-50d88a56dc94-kube-api-access-xvktz\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.440150 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.440128 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9551e061-4d3f-4ede-9d70-50d88a56dc94-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-xr4bj\" (UID: \"9551e061-4d3f-4ede-9d70-50d88a56dc94\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.529303 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529216 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529303 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529258 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529303 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529295 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529575 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529344 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529575 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529400 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95krk\" (UniqueName: \"kubernetes.io/projected/2af08a2d-6765-4d43-a546-9c11fbd32823-kube-api-access-95krk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529575 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529727 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529634 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af08a2d-6765-4d43-a546-9c11fbd32823-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529727 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529727 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529693 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529858 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529725 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.529858 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.529738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.530113 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.530032 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.530113 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.530099 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.530609 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.530587 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2af08a2d-6765-4d43-a546-9c11fbd32823-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.531948 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.531922 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.532034 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.531984 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.570156 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.570131 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95krk\" (UniqueName: \"kubernetes.io/projected/2af08a2d-6765-4d43-a546-9c11fbd32823-kube-api-access-95krk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.576354 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.576331 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2af08a2d-6765-4d43-a546-9c11fbd32823-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45l7t7\" (UID: \"2af08a2d-6765-4d43-a546-9c11fbd32823\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.591639 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.591621 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:39.677155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.677122 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:39.801210 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.801129 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj"] Jul 13 17:26:39.803347 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:26:39.803308 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9551e061_4d3f_4ede_9d70_50d88a56dc94.slice/crio-973f5426d7edbc59f7bf3966fafde40f5913f26286c897ab0ff9ca8335c70b1c WatchSource:0}: Error finding container 973f5426d7edbc59f7bf3966fafde40f5913f26286c897ab0ff9ca8335c70b1c: Status 404 returned error can't find the container with id 973f5426d7edbc59f7bf3966fafde40f5913f26286c897ab0ff9ca8335c70b1c Jul 13 17:26:39.870971 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:39.870947 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7"] Jul 13 17:26:39.873104 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:26:39.873079 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af08a2d_6765_4d43_a546_9c11fbd32823.slice/crio-593e8936b472de1705afc2dc719816a1ac980ce8d19b03a666cd248f751e0dc2 WatchSource:0}: Error finding container 593e8936b472de1705afc2dc719816a1ac980ce8d19b03a666cd248f751e0dc2: Status 404 returned error can't find the container with id 593e8936b472de1705afc2dc719816a1ac980ce8d19b03a666cd248f751e0dc2 Jul 13 17:26:40.685436 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:40.685399 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" event={"ID":"2af08a2d-6765-4d43-a546-9c11fbd32823","Type":"ContainerStarted","Data":"593e8936b472de1705afc2dc719816a1ac980ce8d19b03a666cd248f751e0dc2"} Jul 13 17:26:40.686804 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:40.686778 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" event={"ID":"9551e061-4d3f-4ede-9d70-50d88a56dc94","Type":"ContainerStarted","Data":"973f5426d7edbc59f7bf3966fafde40f5913f26286c897ab0ff9ca8335c70b1c"} Jul 13 17:26:41.815341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:41.815274 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:26:41.815641 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:41.815362 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:26:42.701487 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:42.701443 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" event={"ID":"9551e061-4d3f-4ede-9d70-50d88a56dc94","Type":"ContainerStarted","Data":"32fffbc9dffdb5222aa30e1e2c683d01c767bc5f44af3782a41cfb4e5700c605"} Jul 13 17:26:42.701682 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:42.701612 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:42.703582 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:42.703507 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-xr4bj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 17:26:42.703719 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:42.703586 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" podUID="9551e061-4d3f-4ede-9d70-50d88a56dc94" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 17:26:43.458190 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.458122 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" podStartSLOduration=2.448359834 podStartE2EDuration="4.458096214s" totalImagesPullingTime="2.00973638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:42.800672444 +0000 UTC m=+540.957159328" watchObservedRunningTime="2026-07-13 17:26:43.458096214 +0000 UTC m=+541.614583091" Jul 13 17:26:43.459816 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.459732 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rh45f"] Jul 13 17:26:43.467706 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.467676 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt"] Jul 13 17:26:43.467834 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.467801 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:43.470908 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.470887 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 13 17:26:43.471031 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.470981 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:43.471103 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.471057 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jkr5d\"" Jul 13 17:26:43.473968 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.473946 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-brnl4\"" Jul 13 17:26:43.474066 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.473956 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 13 17:26:43.481762 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.481733 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rh45f"] Jul 13 17:26:43.510433 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.510401 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt"] Jul 13 17:26:43.568215 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.568172 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/436dc897-f260-4a0c-a8b3-fe6aff676d24-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x5ptt\" (UID: \"436dc897-f260-4a0c-a8b3-fe6aff676d24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:43.568417 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.568245 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6tb8\" (UniqueName: \"kubernetes.io/projected/436dc897-f260-4a0c-a8b3-fe6aff676d24-kube-api-access-l6tb8\") pod \"llmisvc-controller-manager-777c6bd7d5-x5ptt\" (UID: \"436dc897-f260-4a0c-a8b3-fe6aff676d24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:43.568417 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.568299 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6lxf\" (UniqueName: \"kubernetes.io/projected/2bca07f1-0dc8-452c-ab58-b49b7479fc07-kube-api-access-x6lxf\") pod \"kserve-controller-manager-d6f4b65b8-rh45f\" (UID: \"2bca07f1-0dc8-452c-ab58-b49b7479fc07\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:43.568417 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.568396 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bca07f1-0dc8-452c-ab58-b49b7479fc07-cert\") pod \"kserve-controller-manager-d6f4b65b8-rh45f\" (UID: \"2bca07f1-0dc8-452c-ab58-b49b7479fc07\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:43.669934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.669897 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6lxf\" (UniqueName: \"kubernetes.io/projected/2bca07f1-0dc8-452c-ab58-b49b7479fc07-kube-api-access-x6lxf\") pod \"kserve-controller-manager-d6f4b65b8-rh45f\" (UID: \"2bca07f1-0dc8-452c-ab58-b49b7479fc07\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:43.670126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.669966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bca07f1-0dc8-452c-ab58-b49b7479fc07-cert\") pod \"kserve-controller-manager-d6f4b65b8-rh45f\" (UID: \"2bca07f1-0dc8-452c-ab58-b49b7479fc07\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:43.670126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.670046 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/436dc897-f260-4a0c-a8b3-fe6aff676d24-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x5ptt\" (UID: \"436dc897-f260-4a0c-a8b3-fe6aff676d24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:43.670126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.670094 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6tb8\" (UniqueName: \"kubernetes.io/projected/436dc897-f260-4a0c-a8b3-fe6aff676d24-kube-api-access-l6tb8\") pod \"llmisvc-controller-manager-777c6bd7d5-x5ptt\" (UID: \"436dc897-f260-4a0c-a8b3-fe6aff676d24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:43.672759 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.672722 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/436dc897-f260-4a0c-a8b3-fe6aff676d24-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-x5ptt\" (UID: \"436dc897-f260-4a0c-a8b3-fe6aff676d24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:43.672889 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.672777 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2bca07f1-0dc8-452c-ab58-b49b7479fc07-cert\") pod \"kserve-controller-manager-d6f4b65b8-rh45f\" (UID: \"2bca07f1-0dc8-452c-ab58-b49b7479fc07\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:43.695932 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.695907 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6lxf\" (UniqueName: \"kubernetes.io/projected/2bca07f1-0dc8-452c-ab58-b49b7479fc07-kube-api-access-x6lxf\") pod \"kserve-controller-manager-d6f4b65b8-rh45f\" (UID: \"2bca07f1-0dc8-452c-ab58-b49b7479fc07\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:43.705771 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.705744 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6tb8\" (UniqueName: \"kubernetes.io/projected/436dc897-f260-4a0c-a8b3-fe6aff676d24-kube-api-access-l6tb8\") pod \"llmisvc-controller-manager-777c6bd7d5-x5ptt\" (UID: \"436dc897-f260-4a0c-a8b3-fe6aff676d24\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:43.707521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.707490 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-xr4bj" Jul 13 17:26:43.781446 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.781377 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:43.786700 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:43.786630 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:44.521759 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.521700 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:26:44.522114 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.521811 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:26:44.522114 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.521855 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:26:44.644652 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:26:44.644617 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bca07f1_0dc8_452c_ab58_b49b7479fc07.slice/crio-a36b79ef00fcd63342bfe0e1bf46189050f87bee5a2bf9977ead782e830533ed WatchSource:0}: Error finding container a36b79ef00fcd63342bfe0e1bf46189050f87bee5a2bf9977ead782e830533ed: Status 404 returned error can't find the container with id a36b79ef00fcd63342bfe0e1bf46189050f87bee5a2bf9977ead782e830533ed Jul 13 17:26:44.648711 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.648688 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-rh45f"] Jul 13 17:26:44.660311 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:26:44.660274 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod436dc897_f260_4a0c_a8b3_fe6aff676d24.slice/crio-d282dbf6e62e2ae35e6c9d3c46aa58a5dc28beb3daee7b9b10201a354db1cb84 WatchSource:0}: Error finding container d282dbf6e62e2ae35e6c9d3c46aa58a5dc28beb3daee7b9b10201a354db1cb84: Status 404 returned error can't find the container with id d282dbf6e62e2ae35e6c9d3c46aa58a5dc28beb3daee7b9b10201a354db1cb84 Jul 13 17:26:44.664486 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.664456 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt"] Jul 13 17:26:44.713848 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.713744 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" event={"ID":"2af08a2d-6765-4d43-a546-9c11fbd32823","Type":"ContainerStarted","Data":"9768b2fd00c28776dbdc783ea59d2f9af8797f2339b0a968465904b757985b1b"} Jul 13 17:26:44.715181 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.715146 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" event={"ID":"436dc897-f260-4a0c-a8b3-fe6aff676d24","Type":"ContainerStarted","Data":"d282dbf6e62e2ae35e6c9d3c46aa58a5dc28beb3daee7b9b10201a354db1cb84"} Jul 13 17:26:44.716562 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.716526 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" event={"ID":"2bca07f1-0dc8-452c-ab58-b49b7479fc07","Type":"ContainerStarted","Data":"a36b79ef00fcd63342bfe0e1bf46189050f87bee5a2bf9977ead782e830533ed"} Jul 13 17:26:44.744429 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:44.744376 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" podStartSLOduration=1.098028799 podStartE2EDuration="5.744359717s" totalImagesPullingTime="4.646330918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:44.740672044 +0000 UTC m=+542.897158918" watchObservedRunningTime="2026-07-13 17:26:44.744359717 +0000 UTC m=+542.900846591" Jul 13 17:26:45.677552 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:45.677515 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:45.683054 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:45.682992 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:45.723065 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:45.723032 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:45.724061 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:45.724038 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45l7t7" Jul 13 17:26:46.669338 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:46.669289 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-lk78q" Jul 13 17:26:48.738647 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:48.738612 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" event={"ID":"436dc897-f260-4a0c-a8b3-fe6aff676d24","Type":"ContainerStarted","Data":"b213a3d18359398f9e3e249af162787811ac172938adea0ed39527bfebb1bb9d"} Jul 13 17:26:48.739088 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:48.738720 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:26:48.740106 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:48.740080 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" event={"ID":"2bca07f1-0dc8-452c-ab58-b49b7479fc07","Type":"ContainerStarted","Data":"7923b8dc28c213caa26f76d494146ac863ba52aa306909636222685134f24bda"} Jul 13 17:26:48.740222 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:48.740127 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:26:48.759047 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:48.759001 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" podStartSLOduration=2.1045596030000002 podStartE2EDuration="5.758987289s" totalImagesPullingTime="3.654427686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:48.757196129 +0000 UTC m=+546.913683004" watchObservedRunningTime="2026-07-13 17:26:48.758987289 +0000 UTC m=+546.915474188" Jul 13 17:26:48.779756 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:26:48.779715 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" podStartSLOduration=2.113633177 podStartE2EDuration="5.779704827s" totalImagesPullingTime="3.66607165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:48.778041026 +0000 UTC m=+546.934527903" watchObservedRunningTime="2026-07-13 17:26:48.779704827 +0000 UTC m=+546.936191700" Jul 13 17:27:19.748672 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:19.748642 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-x5ptt" Jul 13 17:27:19.751641 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:19.751613 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-rh45f" Jul 13 17:27:42.494004 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:42.493972 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:27:42.495522 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:42.495493 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:27:46.768463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.768427 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm"] Jul 13 17:27:46.771025 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.771007 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.773332 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.773289 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 17:27:46.773448 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.773385 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4rw67\"" Jul 13 17:27:46.773490 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.773302 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 17:27:46.780730 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.780708 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm"] Jul 13 17:27:46.878606 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.878563 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.878606 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.878609 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.878848 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.878732 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zs6k\" (UniqueName: \"kubernetes.io/projected/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-kube-api-access-9zs6k\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.979842 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.979805 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.979842 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.979844 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.980090 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.979929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zs6k\" (UniqueName: \"kubernetes.io/projected/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-kube-api-access-9zs6k\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.980278 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.980254 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.980392 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.980281 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:46.988748 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:46.988725 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zs6k\" (UniqueName: \"kubernetes.io/projected/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-kube-api-access-9zs6k\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:47.080809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.080734 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:47.189258 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.189227 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p"] Jul 13 17:27:47.192219 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.192195 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.212168 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.212105 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p"] Jul 13 17:27:47.240967 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:27:47.240927 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5a9bb6_c8b2_4576_9e30_cec9695a7e6f.slice/crio-5e574c9ec2e5d0d0a570fab15795530c9a99522b66ac3bac8ebb6da32ff40958 WatchSource:0}: Error finding container 5e574c9ec2e5d0d0a570fab15795530c9a99522b66ac3bac8ebb6da32ff40958: Status 404 returned error can't find the container with id 5e574c9ec2e5d0d0a570fab15795530c9a99522b66ac3bac8ebb6da32ff40958 Jul 13 17:27:47.240967 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.240937 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm"] Jul 13 17:27:47.283279 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.283251 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.283400 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.283303 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.283469 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.283401 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffg6v\" (UniqueName: \"kubernetes.io/projected/855e3210-cac3-4dcb-a191-2f839797472e-kube-api-access-ffg6v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.384731 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.384698 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.384731 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.384735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.384904 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.384763 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffg6v\" (UniqueName: \"kubernetes.io/projected/855e3210-cac3-4dcb-a191-2f839797472e-kube-api-access-ffg6v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.385094 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.385074 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.385131 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.385093 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.394083 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.394059 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffg6v\" (UniqueName: \"kubernetes.io/projected/855e3210-cac3-4dcb-a191-2f839797472e-kube-api-access-ffg6v\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.502254 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.502223 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:47.560867 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.560836 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s"] Jul 13 17:27:47.563994 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.563973 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.574451 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.574426 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s"] Jul 13 17:27:47.586564 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.586532 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hp7n\" (UniqueName: \"kubernetes.io/projected/51b2cced-8bce-490f-953d-5070575075c7-kube-api-access-5hp7n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.586680 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.586635 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.586680 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.586676 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.645800 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.645777 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p"] Jul 13 17:27:47.647016 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:27:47.646991 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855e3210_cac3_4dcb_a191_2f839797472e.slice/crio-f62572eb555eefa91e8d043f758eccad04855e1c5b6a5d50556071a40b0a2033 WatchSource:0}: Error finding container f62572eb555eefa91e8d043f758eccad04855e1c5b6a5d50556071a40b0a2033: Status 404 returned error can't find the container with id f62572eb555eefa91e8d043f758eccad04855e1c5b6a5d50556071a40b0a2033 Jul 13 17:27:47.687294 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.687268 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hp7n\" (UniqueName: \"kubernetes.io/projected/51b2cced-8bce-490f-953d-5070575075c7-kube-api-access-5hp7n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.687426 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.687362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.687426 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.687409 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.687771 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.687751 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.687857 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.687778 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.697156 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.697127 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hp7n\" (UniqueName: \"kubernetes.io/projected/51b2cced-8bce-490f-953d-5070575075c7-kube-api-access-5hp7n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.876309 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.876278 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:47.964195 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.964161 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x"] Jul 13 17:27:47.969521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.969499 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:47.974919 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:47.974883 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x"] Jul 13 17:27:48.013300 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.013274 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s"] Jul 13 17:27:48.014756 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:27:48.014727 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b2cced_8bce_490f_953d_5070575075c7.slice/crio-0e312c7962cbb856608956f412f0673cfd7c21f48358999470e4bbbf4abbce24 WatchSource:0}: Error finding container 0e312c7962cbb856608956f412f0673cfd7c21f48358999470e4bbbf4abbce24: Status 404 returned error can't find the container with id 0e312c7962cbb856608956f412f0673cfd7c21f48358999470e4bbbf4abbce24 Jul 13 17:27:48.020987 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.020875 2579 generic.go:356] "Generic (PLEG): container finished" podID="855e3210-cac3-4dcb-a191-2f839797472e" containerID="054140f81f131531d16fcb4342fe82753cf89d66e54e74620a73811dfb51feb5" exitCode=0 Jul 13 17:27:48.020987 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.020921 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" event={"ID":"855e3210-cac3-4dcb-a191-2f839797472e","Type":"ContainerDied","Data":"054140f81f131531d16fcb4342fe82753cf89d66e54e74620a73811dfb51feb5"} Jul 13 17:27:48.021170 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.021052 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" event={"ID":"855e3210-cac3-4dcb-a191-2f839797472e","Type":"ContainerStarted","Data":"f62572eb555eefa91e8d043f758eccad04855e1c5b6a5d50556071a40b0a2033"} Jul 13 17:27:48.023051 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.023030 2579 generic.go:356] "Generic (PLEG): container finished" podID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerID="ebd8b8a302c00ee46162df470114cbcf6f810e52730434fc9e18935abb7b4c5c" exitCode=0 Jul 13 17:27:48.023136 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.023108 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" event={"ID":"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f","Type":"ContainerDied","Data":"ebd8b8a302c00ee46162df470114cbcf6f810e52730434fc9e18935abb7b4c5c"} Jul 13 17:27:48.023184 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.023143 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" event={"ID":"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f","Type":"ContainerStarted","Data":"5e574c9ec2e5d0d0a570fab15795530c9a99522b66ac3bac8ebb6da32ff40958"} Jul 13 17:27:48.091596 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.091568 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tcx7\" (UniqueName: \"kubernetes.io/projected/d074c6fb-0915-410c-81ca-39303c43cecf-kube-api-access-2tcx7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.091740 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.091618 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.091814 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.091788 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.192577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.192508 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.192577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.192564 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tcx7\" (UniqueName: \"kubernetes.io/projected/d074c6fb-0915-410c-81ca-39303c43cecf-kube-api-access-2tcx7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.192733 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.192608 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.192863 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.192843 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.192962 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.192941 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.202093 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.202068 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tcx7\" (UniqueName: \"kubernetes.io/projected/d074c6fb-0915-410c-81ca-39303c43cecf-kube-api-access-2tcx7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.282088 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.282060 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:48.410332 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:48.410293 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x"] Jul 13 17:27:48.411892 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:27:48.411865 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd074c6fb_0915_410c_81ca_39303c43cecf.slice/crio-ec3c5b9d3779b9b083ff4e3fb7c61cd481faa3b8d4a2547fa9a262e191f58b14 WatchSource:0}: Error finding container ec3c5b9d3779b9b083ff4e3fb7c61cd481faa3b8d4a2547fa9a262e191f58b14: Status 404 returned error can't find the container with id ec3c5b9d3779b9b083ff4e3fb7c61cd481faa3b8d4a2547fa9a262e191f58b14 Jul 13 17:27:49.030451 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:49.030373 2579 generic.go:356] "Generic (PLEG): container finished" podID="51b2cced-8bce-490f-953d-5070575075c7" containerID="f4b0bfa211a8733ef45a52a0c96a7baf2173034fa8c45d0d2597e9c23cc08f99" exitCode=0 Jul 13 17:27:49.030925 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:49.030459 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" event={"ID":"51b2cced-8bce-490f-953d-5070575075c7","Type":"ContainerDied","Data":"f4b0bfa211a8733ef45a52a0c96a7baf2173034fa8c45d0d2597e9c23cc08f99"} Jul 13 17:27:49.030925 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:49.030523 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" event={"ID":"51b2cced-8bce-490f-953d-5070575075c7","Type":"ContainerStarted","Data":"0e312c7962cbb856608956f412f0673cfd7c21f48358999470e4bbbf4abbce24"} Jul 13 17:27:49.032401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:49.032380 2579 generic.go:356] "Generic (PLEG): container finished" podID="855e3210-cac3-4dcb-a191-2f839797472e" containerID="447bf3a1237060b5e162b47314d3d9938120ed6fe27722ccc2d77f3bd566033c" exitCode=0 Jul 13 17:27:49.032517 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:49.032458 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" event={"ID":"855e3210-cac3-4dcb-a191-2f839797472e","Type":"ContainerDied","Data":"447bf3a1237060b5e162b47314d3d9938120ed6fe27722ccc2d77f3bd566033c"} Jul 13 17:27:49.033973 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:49.033952 2579 generic.go:356] "Generic (PLEG): container finished" podID="d074c6fb-0915-410c-81ca-39303c43cecf" containerID="919a3592ed39545972daca5ced70e5b02ed8dd628b461b48244a0d85bf811021" exitCode=0 Jul 13 17:27:49.034074 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:49.034029 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" event={"ID":"d074c6fb-0915-410c-81ca-39303c43cecf","Type":"ContainerDied","Data":"919a3592ed39545972daca5ced70e5b02ed8dd628b461b48244a0d85bf811021"} Jul 13 17:27:49.034136 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:49.034076 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" event={"ID":"d074c6fb-0915-410c-81ca-39303c43cecf","Type":"ContainerStarted","Data":"ec3c5b9d3779b9b083ff4e3fb7c61cd481faa3b8d4a2547fa9a262e191f58b14"} Jul 13 17:27:49.174130 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:27:49.174100 2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5a9bb6_c8b2_4576_9e30_cec9695a7e6f.slice/crio-1c845ef08901f0be8bfde3a44c17f527768431f34a7c93ab2cd13514f1255999.scope\": RecentStats: unable to find data in memory cache]" Jul 13 17:27:50.040738 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:50.040708 2579 generic.go:356] "Generic (PLEG): container finished" podID="51b2cced-8bce-490f-953d-5070575075c7" containerID="6ba2d9402879627add1f85b1179925b264f6ba7f6ab1236e393c88889d5a902e" exitCode=0 Jul 13 17:27:50.041178 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:50.040798 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" event={"ID":"51b2cced-8bce-490f-953d-5070575075c7","Type":"ContainerDied","Data":"6ba2d9402879627add1f85b1179925b264f6ba7f6ab1236e393c88889d5a902e"} Jul 13 17:27:50.043107 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:50.043084 2579 generic.go:356] "Generic (PLEG): container finished" podID="855e3210-cac3-4dcb-a191-2f839797472e" containerID="ecb69890a21a1aa2e7aae3f0bd5ffd55946e9092474711bb0e1c874a335fb507" exitCode=0 Jul 13 17:27:50.043187 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:50.043161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" event={"ID":"855e3210-cac3-4dcb-a191-2f839797472e","Type":"ContainerDied","Data":"ecb69890a21a1aa2e7aae3f0bd5ffd55946e9092474711bb0e1c874a335fb507"} Jul 13 17:27:50.044786 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:50.044764 2579 generic.go:356] "Generic (PLEG): container finished" podID="d074c6fb-0915-410c-81ca-39303c43cecf" containerID="b7748015d38b23fc050a0b854d45d44a058dba94e42500e3c76fbb5e648614a0" exitCode=0 Jul 13 17:27:50.044880 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:50.044845 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" event={"ID":"d074c6fb-0915-410c-81ca-39303c43cecf","Type":"ContainerDied","Data":"b7748015d38b23fc050a0b854d45d44a058dba94e42500e3c76fbb5e648614a0"} Jul 13 17:27:50.046940 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:50.046922 2579 generic.go:356] "Generic (PLEG): container finished" podID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerID="1c845ef08901f0be8bfde3a44c17f527768431f34a7c93ab2cd13514f1255999" exitCode=0 Jul 13 17:27:50.047050 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:50.046993 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" event={"ID":"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f","Type":"ContainerDied","Data":"1c845ef08901f0be8bfde3a44c17f527768431f34a7c93ab2cd13514f1255999"} Jul 13 17:27:51.054543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.054505 2579 generic.go:356] "Generic (PLEG): container finished" podID="51b2cced-8bce-490f-953d-5070575075c7" containerID="9bde5f729fc5159e6f98262447c9aa0df82f81ba5a478e1aa462d8c67aa3d2cb" exitCode=0 Jul 13 17:27:51.054886 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.054592 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" event={"ID":"51b2cced-8bce-490f-953d-5070575075c7","Type":"ContainerDied","Data":"9bde5f729fc5159e6f98262447c9aa0df82f81ba5a478e1aa462d8c67aa3d2cb"} Jul 13 17:27:51.056604 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.056583 2579 generic.go:356] "Generic (PLEG): container finished" podID="d074c6fb-0915-410c-81ca-39303c43cecf" containerID="d0144a735853d33b79f9f778772efef90ab5955b3159f0dd733df180a65b1719" exitCode=0 Jul 13 17:27:51.056712 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.056657 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" event={"ID":"d074c6fb-0915-410c-81ca-39303c43cecf","Type":"ContainerDied","Data":"d0144a735853d33b79f9f778772efef90ab5955b3159f0dd733df180a65b1719"} Jul 13 17:27:51.058701 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.058680 2579 generic.go:356] "Generic (PLEG): container finished" podID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerID="4491b511cf1105cc69038ae3b35bee75ce7a462500ea7ec48d760f5aded2cd77" exitCode=0 Jul 13 17:27:51.058803 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.058747 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" event={"ID":"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f","Type":"ContainerDied","Data":"4491b511cf1105cc69038ae3b35bee75ce7a462500ea7ec48d760f5aded2cd77"} Jul 13 17:27:51.183231 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.183205 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:51.222172 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.222144 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-util\") pod \"855e3210-cac3-4dcb-a191-2f839797472e\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " Jul 13 17:27:51.222283 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.222197 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-bundle\") pod \"855e3210-cac3-4dcb-a191-2f839797472e\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " Jul 13 17:27:51.222362 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.222345 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffg6v\" (UniqueName: \"kubernetes.io/projected/855e3210-cac3-4dcb-a191-2f839797472e-kube-api-access-ffg6v\") pod \"855e3210-cac3-4dcb-a191-2f839797472e\" (UID: \"855e3210-cac3-4dcb-a191-2f839797472e\") " Jul 13 17:27:51.222735 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.222701 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-bundle" (OuterVolumeSpecName: "bundle") pod "855e3210-cac3-4dcb-a191-2f839797472e" (UID: "855e3210-cac3-4dcb-a191-2f839797472e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:27:51.224605 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.224577 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/855e3210-cac3-4dcb-a191-2f839797472e-kube-api-access-ffg6v" (OuterVolumeSpecName: "kube-api-access-ffg6v") pod "855e3210-cac3-4dcb-a191-2f839797472e" (UID: "855e3210-cac3-4dcb-a191-2f839797472e"). InnerVolumeSpecName "kube-api-access-ffg6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:27:51.227771 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.227749 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-util" (OuterVolumeSpecName: "util") pod "855e3210-cac3-4dcb-a191-2f839797472e" (UID: "855e3210-cac3-4dcb-a191-2f839797472e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:27:51.323519 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.323456 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:51.323519 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.323480 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffg6v\" (UniqueName: \"kubernetes.io/projected/855e3210-cac3-4dcb-a191-2f839797472e-kube-api-access-ffg6v\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:51.323519 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:51.323491 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/855e3210-cac3-4dcb-a191-2f839797472e-util\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.066335 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.066304 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" Jul 13 17:27:52.066772 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.066345 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p" event={"ID":"855e3210-cac3-4dcb-a191-2f839797472e","Type":"ContainerDied","Data":"f62572eb555eefa91e8d043f758eccad04855e1c5b6a5d50556071a40b0a2033"} Jul 13 17:27:52.066772 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.066385 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f62572eb555eefa91e8d043f758eccad04855e1c5b6a5d50556071a40b0a2033" Jul 13 17:27:52.220932 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.220902 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:52.264899 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.264872 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:52.267853 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.267831 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:52.331726 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331651 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-bundle\") pod \"51b2cced-8bce-490f-953d-5070575075c7\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " Jul 13 17:27:52.331726 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331706 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tcx7\" (UniqueName: \"kubernetes.io/projected/d074c6fb-0915-410c-81ca-39303c43cecf-kube-api-access-2tcx7\") pod \"d074c6fb-0915-410c-81ca-39303c43cecf\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " Jul 13 17:27:52.331934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331749 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-bundle\") pod \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " Jul 13 17:27:52.331934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331778 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-util\") pod \"51b2cced-8bce-490f-953d-5070575075c7\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " Jul 13 17:27:52.331934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331797 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-bundle\") pod \"d074c6fb-0915-410c-81ca-39303c43cecf\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " Jul 13 17:27:52.331934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331832 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zs6k\" (UniqueName: \"kubernetes.io/projected/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-kube-api-access-9zs6k\") pod \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " Jul 13 17:27:52.331934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331865 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5hp7n\" (UniqueName: \"kubernetes.io/projected/51b2cced-8bce-490f-953d-5070575075c7-kube-api-access-5hp7n\") pod \"51b2cced-8bce-490f-953d-5070575075c7\" (UID: \"51b2cced-8bce-490f-953d-5070575075c7\") " Jul 13 17:27:52.331934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331885 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-util\") pod \"d074c6fb-0915-410c-81ca-39303c43cecf\" (UID: \"d074c6fb-0915-410c-81ca-39303c43cecf\") " Jul 13 17:27:52.331934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.331920 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-util\") pod \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\" (UID: \"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f\") " Jul 13 17:27:52.332286 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.332232 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-bundle" (OuterVolumeSpecName: "bundle") pod "51b2cced-8bce-490f-953d-5070575075c7" (UID: "51b2cced-8bce-490f-953d-5070575075c7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:27:52.332383 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.332363 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-bundle" (OuterVolumeSpecName: "bundle") pod "6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" (UID: "6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:27:52.333026 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.332996 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-bundle" (OuterVolumeSpecName: "bundle") pod "d074c6fb-0915-410c-81ca-39303c43cecf" (UID: "d074c6fb-0915-410c-81ca-39303c43cecf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:27:52.334225 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.334186 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d074c6fb-0915-410c-81ca-39303c43cecf-kube-api-access-2tcx7" (OuterVolumeSpecName: "kube-api-access-2tcx7") pod "d074c6fb-0915-410c-81ca-39303c43cecf" (UID: "d074c6fb-0915-410c-81ca-39303c43cecf"). InnerVolumeSpecName "kube-api-access-2tcx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:27:52.334356 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.334313 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-kube-api-access-9zs6k" (OuterVolumeSpecName: "kube-api-access-9zs6k") pod "6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" (UID: "6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f"). InnerVolumeSpecName "kube-api-access-9zs6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:27:52.334786 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.334761 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51b2cced-8bce-490f-953d-5070575075c7-kube-api-access-5hp7n" (OuterVolumeSpecName: "kube-api-access-5hp7n") pod "51b2cced-8bce-490f-953d-5070575075c7" (UID: "51b2cced-8bce-490f-953d-5070575075c7"). InnerVolumeSpecName "kube-api-access-5hp7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:27:52.339375 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.339353 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-util" (OuterVolumeSpecName: "util") pod "6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" (UID: "6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:27:52.340935 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.340908 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-util" (OuterVolumeSpecName: "util") pod "51b2cced-8bce-490f-953d-5070575075c7" (UID: "51b2cced-8bce-490f-953d-5070575075c7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:27:52.341457 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.341438 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-util" (OuterVolumeSpecName: "util") pod "d074c6fb-0915-410c-81ca-39303c43cecf" (UID: "d074c6fb-0915-410c-81ca-39303c43cecf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:27:52.432692 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432663 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.432692 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432689 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tcx7\" (UniqueName: \"kubernetes.io/projected/d074c6fb-0915-410c-81ca-39303c43cecf-kube-api-access-2tcx7\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.432844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432700 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.432844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432708 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51b2cced-8bce-490f-953d-5070575075c7-util\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.432844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432716 2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.432844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432724 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zs6k\" (UniqueName: \"kubernetes.io/projected/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-kube-api-access-9zs6k\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.432844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432732 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hp7n\" (UniqueName: \"kubernetes.io/projected/51b2cced-8bce-490f-953d-5070575075c7-kube-api-access-5hp7n\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.432844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432741 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d074c6fb-0915-410c-81ca-39303c43cecf-util\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:52.432844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:52.432748 2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f-util\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:27:53.073016 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.072924 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" Jul 13 17:27:53.073016 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.072927 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s" event={"ID":"51b2cced-8bce-490f-953d-5070575075c7","Type":"ContainerDied","Data":"0e312c7962cbb856608956f412f0673cfd7c21f48358999470e4bbbf4abbce24"} Jul 13 17:27:53.073550 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.073089 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e312c7962cbb856608956f412f0673cfd7c21f48358999470e4bbbf4abbce24" Jul 13 17:27:53.075169 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.075140 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" event={"ID":"d074c6fb-0915-410c-81ca-39303c43cecf","Type":"ContainerDied","Data":"ec3c5b9d3779b9b083ff4e3fb7c61cd481faa3b8d4a2547fa9a262e191f58b14"} Jul 13 17:27:53.075300 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.075188 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec3c5b9d3779b9b083ff4e3fb7c61cd481faa3b8d4a2547fa9a262e191f58b14" Jul 13 17:27:53.075300 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.075196 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x" Jul 13 17:27:53.077288 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.077256 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" event={"ID":"6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f","Type":"ContainerDied","Data":"5e574c9ec2e5d0d0a570fab15795530c9a99522b66ac3bac8ebb6da32ff40958"} Jul 13 17:27:53.077288 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.077279 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm" Jul 13 17:27:53.077451 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:27:53.077298 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e574c9ec2e5d0d0a570fab15795530c9a99522b66ac3bac8ebb6da32ff40958" Jul 13 17:28:09.588118 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588020 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt"] Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588569 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d074c6fb-0915-410c-81ca-39303c43cecf" containerName="extract" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588590 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d074c6fb-0915-410c-81ca-39303c43cecf" containerName="extract" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588607 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855e3210-cac3-4dcb-a191-2f839797472e" containerName="util" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588617 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="855e3210-cac3-4dcb-a191-2f839797472e" containerName="util" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588628 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerName="pull" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588637 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerName="pull" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588650 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerName="util" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588659 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerName="util" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588670 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b2cced-8bce-490f-953d-5070575075c7" containerName="util" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588677 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b2cced-8bce-490f-953d-5070575075c7" containerName="util" Jul 13 17:28:09.588683 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588686 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d074c6fb-0915-410c-81ca-39303c43cecf" containerName="pull" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588696 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d074c6fb-0915-410c-81ca-39303c43cecf" containerName="pull" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588706 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855e3210-cac3-4dcb-a191-2f839797472e" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588713 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="855e3210-cac3-4dcb-a191-2f839797472e" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588723 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b2cced-8bce-490f-953d-5070575075c7" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588732 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b2cced-8bce-490f-953d-5070575075c7" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588745 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b2cced-8bce-490f-953d-5070575075c7" containerName="pull" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588753 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b2cced-8bce-490f-953d-5070575075c7" containerName="pull" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588788 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d074c6fb-0915-410c-81ca-39303c43cecf" containerName="util" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588796 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d074c6fb-0915-410c-81ca-39303c43cecf" containerName="util" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588805 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="855e3210-cac3-4dcb-a191-2f839797472e" containerName="pull" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588812 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="855e3210-cac3-4dcb-a191-2f839797472e" containerName="pull" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588825 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588833 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588920 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="51b2cced-8bce-490f-953d-5070575075c7" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588934 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="855e3210-cac3-4dcb-a191-2f839797472e" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588944 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d074c6fb-0915-410c-81ca-39303c43cecf" containerName="extract" Jul 13 17:28:09.589201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.588954 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f" containerName="extract" Jul 13 17:28:09.592386 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.592363 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:09.594754 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.594729 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 17:28:09.594878 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.594831 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t6clk\"" Jul 13 17:28:09.594958 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.594892 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 17:28:09.607538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.607516 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt"] Jul 13 17:28:09.688396 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.688372 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d7n8\" (UniqueName: \"kubernetes.io/projected/3cd3db13-b98f-4414-8c12-6f9df726d257-kube-api-access-9d7n8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" (UID: \"3cd3db13-b98f-4414-8c12-6f9df726d257\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:09.688498 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.688411 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cd3db13-b98f-4414-8c12-6f9df726d257-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" (UID: \"3cd3db13-b98f-4414-8c12-6f9df726d257\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:09.788941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.788916 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d7n8\" (UniqueName: \"kubernetes.io/projected/3cd3db13-b98f-4414-8c12-6f9df726d257-kube-api-access-9d7n8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" (UID: \"3cd3db13-b98f-4414-8c12-6f9df726d257\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:09.789035 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.788955 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cd3db13-b98f-4414-8c12-6f9df726d257-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" (UID: \"3cd3db13-b98f-4414-8c12-6f9df726d257\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:09.789274 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.789258 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cd3db13-b98f-4414-8c12-6f9df726d257-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" (UID: \"3cd3db13-b98f-4414-8c12-6f9df726d257\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:09.800516 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.800492 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d7n8\" (UniqueName: \"kubernetes.io/projected/3cd3db13-b98f-4414-8c12-6f9df726d257-kube-api-access-9d7n8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" (UID: \"3cd3db13-b98f-4414-8c12-6f9df726d257\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:09.903199 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:09.903175 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:10.041826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:10.041794 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt"] Jul 13 17:28:10.043626 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:28:10.043598 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cd3db13_b98f_4414_8c12_6f9df726d257.slice/crio-76c718c952151b38a88570e2f9a26abb4fad2fc837d36c4b0230960caafddf15 WatchSource:0}: Error finding container 76c718c952151b38a88570e2f9a26abb4fad2fc837d36c4b0230960caafddf15: Status 404 returned error can't find the container with id 76c718c952151b38a88570e2f9a26abb4fad2fc837d36c4b0230960caafddf15 Jul 13 17:28:10.171472 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:10.171411 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" event={"ID":"3cd3db13-b98f-4414-8c12-6f9df726d257","Type":"ContainerStarted","Data":"76c718c952151b38a88570e2f9a26abb4fad2fc837d36c4b0230960caafddf15"} Jul 13 17:28:17.213569 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:17.213534 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" event={"ID":"3cd3db13-b98f-4414-8c12-6f9df726d257","Type":"ContainerStarted","Data":"dccd46f8d39804bc1eb880fe40f601e822320f0a9d41fdfd64b8387d7742a65b"} Jul 13 17:28:17.213895 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:17.213735 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:17.242782 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:17.239575 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" podStartSLOduration=1.269751856 podStartE2EDuration="8.239542399s" totalImagesPullingTime="6.969790543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:28:17.236974227 +0000 UTC m=+635.393461102" watchObservedRunningTime="2026-07-13 17:28:17.239542399 +0000 UTC m=+635.396029274" Jul 13 17:28:28.221278 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:28.221243 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:29.681623 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.681585 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq"] Jul 13 17:28:29.686643 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.686604 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:29.698236 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.698210 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq"] Jul 13 17:28:29.759932 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.759896 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b48321-812f-4a99-8bf9-d95c191c13ff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" (UID: \"70b48321-812f-4a99-8bf9-d95c191c13ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:29.760096 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.759947 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l45q\" (UniqueName: \"kubernetes.io/projected/70b48321-812f-4a99-8bf9-d95c191c13ff-kube-api-access-8l45q\") pod \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" (UID: \"70b48321-812f-4a99-8bf9-d95c191c13ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:29.861182 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.861145 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l45q\" (UniqueName: \"kubernetes.io/projected/70b48321-812f-4a99-8bf9-d95c191c13ff-kube-api-access-8l45q\") pod \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" (UID: \"70b48321-812f-4a99-8bf9-d95c191c13ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:29.861364 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.861233 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b48321-812f-4a99-8bf9-d95c191c13ff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" (UID: \"70b48321-812f-4a99-8bf9-d95c191c13ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:29.861578 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.861561 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b48321-812f-4a99-8bf9-d95c191c13ff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" (UID: \"70b48321-812f-4a99-8bf9-d95c191c13ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:29.874479 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.874454 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l45q\" (UniqueName: \"kubernetes.io/projected/70b48321-812f-4a99-8bf9-d95c191c13ff-kube-api-access-8l45q\") pod \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" (UID: \"70b48321-812f-4a99-8bf9-d95c191c13ff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:29.997981 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:29.997898 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:30.110851 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.110815 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq"] Jul 13 17:28:30.128034 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.128005 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq"] Jul 13 17:28:30.165790 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.165757 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt"] Jul 13 17:28:30.165999 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.165954 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" containerName="manager" containerID="cri-o://dccd46f8d39804bc1eb880fe40f601e822320f0a9d41fdfd64b8387d7742a65b" gracePeriod=2 Jul 13 17:28:30.173678 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.173611 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt"] Jul 13 17:28:30.199016 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.198985 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp"] Jul 13 17:28:30.199492 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.199465 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" containerName="manager" Jul 13 17:28:30.199492 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.199485 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" containerName="manager" Jul 13 17:28:30.199656 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.199608 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" containerName="manager" Jul 13 17:28:30.203023 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.203002 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:30.205603 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.205565 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:30.236261 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.236228 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp"] Jul 13 17:28:30.265460 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.265382 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkj4t\" (UniqueName: \"kubernetes.io/projected/93596795-3e4a-41a3-aeea-bfe300ad3b03-kube-api-access-xkj4t\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9fnrp\" (UID: \"93596795-3e4a-41a3-aeea-bfe300ad3b03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:30.265460 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.265427 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93596795-3e4a-41a3-aeea-bfe300ad3b03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9fnrp\" (UID: \"93596795-3e4a-41a3-aeea-bfe300ad3b03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:30.270759 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.270697 2579 log.go:32] "RunPodSandbox from runtime service failed" err=< Jul 13 17:28:30.270759 ip-10-0-143-131 kubenswrapper[2579]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5zvdq_kuadrant-system_70b48321-812f-4a99-8bf9-d95c191c13ff_0(c6a144089f5023453ccbd255e4415f73cb9424ce7cf151095af490860d36ec38): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5zvdq to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"c6a144089f5023453ccbd255e4415f73cb9424ce7cf151095af490860d36ec38" Netns:"/var/run/netns/146bac72-fb0a-4d35-aaf1-d6ceeffe5bd8" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-5zvdq;K8S_POD_INFRA_CONTAINER_ID=c6a144089f5023453ccbd255e4415f73cb9424ce7cf151095af490860d36ec38;K8S_POD_UID=70b48321-812f-4a99-8bf9-d95c191c13ff" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq/70b48321-812f-4a99-8bf9-d95c191c13ff]: error setting the networks status: SetPodNetworkStatusAnnotation: failed to update the pod kuadrant-operator-controller-manager-55467d9f7-5zvdq in out of cluster comm: SetNetworkStatus: failed to update the pod kuadrant-operator-controller-manager-55467d9f7-5zvdq in out of cluster comm: status update failed for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq: pods "kuadrant-operator-controller-manager-55467d9f7-5zvdq" not found Jul 13 17:28:30.270759 ip-10-0-143-131 kubenswrapper[2579]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 13 17:28:30.270759 ip-10-0-143-131 kubenswrapper[2579]: > Jul 13 17:28:30.270963 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.270782 2579 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Jul 13 17:28:30.270963 ip-10-0-143-131 kubenswrapper[2579]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-5zvdq_kuadrant-system_70b48321-812f-4a99-8bf9-d95c191c13ff_0(c6a144089f5023453ccbd255e4415f73cb9424ce7cf151095af490860d36ec38): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-5zvdq to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"c6a144089f5023453ccbd255e4415f73cb9424ce7cf151095af490860d36ec38" Netns:"/var/run/netns/146bac72-fb0a-4d35-aaf1-d6ceeffe5bd8" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-5zvdq;K8S_POD_INFRA_CONTAINER_ID=c6a144089f5023453ccbd255e4415f73cb9424ce7cf151095af490860d36ec38;K8S_POD_UID=70b48321-812f-4a99-8bf9-d95c191c13ff" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq] networking: Multus: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq/70b48321-812f-4a99-8bf9-d95c191c13ff]: error setting the networks status: SetPodNetworkStatusAnnotation: failed to update the pod kuadrant-operator-controller-manager-55467d9f7-5zvdq in out of cluster comm: SetNetworkStatus: failed to update the pod kuadrant-operator-controller-manager-55467d9f7-5zvdq in out of cluster comm: status update failed for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq: pods "kuadrant-operator-controller-manager-55467d9f7-5zvdq" not found Jul 13 17:28:30.270963 ip-10-0-143-131 kubenswrapper[2579]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 13 17:28:30.270963 ip-10-0-143-131 kubenswrapper[2579]: > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:30.284624 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.284600 2579 generic.go:356] "Generic (PLEG): container finished" podID="3cd3db13-b98f-4414-8c12-6f9df726d257" containerID="dccd46f8d39804bc1eb880fe40f601e822320f0a9d41fdfd64b8387d7742a65b" exitCode=0 Jul 13 17:28:30.284720 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.284677 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:30.286847 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.286818 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:30.288932 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.288911 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="70b48321-812f-4a99-8bf9-d95c191c13ff" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:30.291205 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.291181 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:30.293241 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.293210 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="70b48321-812f-4a99-8bf9-d95c191c13ff" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:30.294773 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.294748 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:30.365920 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.365892 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l45q\" (UniqueName: \"kubernetes.io/projected/70b48321-812f-4a99-8bf9-d95c191c13ff-kube-api-access-8l45q\") pod \"70b48321-812f-4a99-8bf9-d95c191c13ff\" (UID: \"70b48321-812f-4a99-8bf9-d95c191c13ff\") " Jul 13 17:28:30.366043 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.365976 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b48321-812f-4a99-8bf9-d95c191c13ff-extensions-socket-volume\") pod \"70b48321-812f-4a99-8bf9-d95c191c13ff\" (UID: \"70b48321-812f-4a99-8bf9-d95c191c13ff\") " Jul 13 17:28:30.366101 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.366087 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkj4t\" (UniqueName: \"kubernetes.io/projected/93596795-3e4a-41a3-aeea-bfe300ad3b03-kube-api-access-xkj4t\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9fnrp\" (UID: \"93596795-3e4a-41a3-aeea-bfe300ad3b03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:30.366144 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.366121 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93596795-3e4a-41a3-aeea-bfe300ad3b03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9fnrp\" (UID: \"93596795-3e4a-41a3-aeea-bfe300ad3b03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:30.366296 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.366266 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70b48321-812f-4a99-8bf9-d95c191c13ff-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "70b48321-812f-4a99-8bf9-d95c191c13ff" (UID: "70b48321-812f-4a99-8bf9-d95c191c13ff"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:28:30.366485 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.366467 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93596795-3e4a-41a3-aeea-bfe300ad3b03-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9fnrp\" (UID: \"93596795-3e4a-41a3-aeea-bfe300ad3b03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:30.369023 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.368999 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70b48321-812f-4a99-8bf9-d95c191c13ff-kube-api-access-8l45q" (OuterVolumeSpecName: "kube-api-access-8l45q") pod "70b48321-812f-4a99-8bf9-d95c191c13ff" (UID: "70b48321-812f-4a99-8bf9-d95c191c13ff"). InnerVolumeSpecName "kube-api-access-8l45q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:28:30.374557 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.374540 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkj4t\" (UniqueName: \"kubernetes.io/projected/93596795-3e4a-41a3-aeea-bfe300ad3b03-kube-api-access-xkj4t\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9fnrp\" (UID: \"93596795-3e4a-41a3-aeea-bfe300ad3b03\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:30.396408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.396386 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:30.398537 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.398507 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rwjkt\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:30.400871 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:30.400849 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="70b48321-812f-4a99-8bf9-d95c191c13ff" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:30.467486 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.467455 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d7n8\" (UniqueName: \"kubernetes.io/projected/3cd3db13-b98f-4414-8c12-6f9df726d257-kube-api-access-9d7n8\") pod \"3cd3db13-b98f-4414-8c12-6f9df726d257\" (UID: \"3cd3db13-b98f-4414-8c12-6f9df726d257\") " Jul 13 17:28:30.467590 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.467518 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cd3db13-b98f-4414-8c12-6f9df726d257-extensions-socket-volume\") pod \"3cd3db13-b98f-4414-8c12-6f9df726d257\" (UID: \"3cd3db13-b98f-4414-8c12-6f9df726d257\") " Jul 13 17:28:30.467706 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.467694 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/70b48321-812f-4a99-8bf9-d95c191c13ff-extensions-socket-volume\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:28:30.467777 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.467706 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l45q\" (UniqueName: \"kubernetes.io/projected/70b48321-812f-4a99-8bf9-d95c191c13ff-kube-api-access-8l45q\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:28:30.468067 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.468032 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cd3db13-b98f-4414-8c12-6f9df726d257-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3cd3db13-b98f-4414-8c12-6f9df726d257" (UID: "3cd3db13-b98f-4414-8c12-6f9df726d257"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:28:30.469467 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.469441 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cd3db13-b98f-4414-8c12-6f9df726d257-kube-api-access-9d7n8" (OuterVolumeSpecName: "kube-api-access-9d7n8") pod "3cd3db13-b98f-4414-8c12-6f9df726d257" (UID: "3cd3db13-b98f-4414-8c12-6f9df726d257"). InnerVolumeSpecName "kube-api-access-9d7n8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:28:30.537662 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.537604 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:30.568598 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.568560 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d7n8\" (UniqueName: \"kubernetes.io/projected/3cd3db13-b98f-4414-8c12-6f9df726d257-kube-api-access-9d7n8\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:28:30.568598 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.568599 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cd3db13-b98f-4414-8c12-6f9df726d257-extensions-socket-volume\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:28:30.661521 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.661492 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cd3db13-b98f-4414-8c12-6f9df726d257" path="/var/lib/kubelet/pods/3cd3db13-b98f-4414-8c12-6f9df726d257/volumes" Jul 13 17:28:30.661862 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.661849 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70b48321-812f-4a99-8bf9-d95c191c13ff" path="/var/lib/kubelet/pods/70b48321-812f-4a99-8bf9-d95c191c13ff/volumes" Jul 13 17:28:30.671968 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:30.671944 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp"] Jul 13 17:28:30.673300 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:28:30.673278 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93596795_3e4a_41a3_aeea_bfe300ad3b03.slice/crio-3459a7fa6dc2e183df359d53dae7783ea93b8df9bde787a24983318620f4fe3d WatchSource:0}: Error finding container 3459a7fa6dc2e183df359d53dae7783ea93b8df9bde787a24983318620f4fe3d: Status 404 returned error can't find the container with id 3459a7fa6dc2e183df359d53dae7783ea93b8df9bde787a24983318620f4fe3d Jul 13 17:28:31.291416 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:31.291337 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rwjkt" Jul 13 17:28:31.291416 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:31.291366 2579 scope.go:117] "RemoveContainer" containerID="dccd46f8d39804bc1eb880fe40f601e822320f0a9d41fdfd64b8387d7742a65b" Jul 13 17:28:31.293560 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:31.293395 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:31.293560 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:31.293416 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" event={"ID":"93596795-3e4a-41a3-aeea-bfe300ad3b03","Type":"ContainerStarted","Data":"b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962"} Jul 13 17:28:31.293560 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:31.293469 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" event={"ID":"93596795-3e4a-41a3-aeea-bfe300ad3b03","Type":"ContainerStarted","Data":"3459a7fa6dc2e183df359d53dae7783ea93b8df9bde787a24983318620f4fe3d"} Jul 13 17:28:31.293560 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:31.293512 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:31.323755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:31.323702 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" podStartSLOduration=1.323684847 podStartE2EDuration="1.323684847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:28:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:28:31.321973425 +0000 UTC m=+649.478460298" watchObservedRunningTime="2026-07-13 17:28:31.323684847 +0000 UTC m=+649.480171723" Jul 13 17:28:31.323953 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:31.323925 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="70b48321-812f-4a99-8bf9-d95c191c13ff" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:32.660585 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:32.660549 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-5zvdq\" is forbidden: User \"system:node:ip-10-0-143-131.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-131.ec2.internal' and this object" podUID="70b48321-812f-4a99-8bf9-d95c191c13ff" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-5zvdq" Jul 13 17:28:42.302621 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:42.302569 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:47.493953 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.493911 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp"] Jul 13 17:28:47.494352 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.494102 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" podUID="93596795-3e4a-41a3-aeea-bfe300ad3b03" containerName="manager" containerID="cri-o://b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962" gracePeriod=10 Jul 13 17:28:47.750158 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.750089 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6"] Jul 13 17:28:47.753675 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.753652 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:47.774018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.773994 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6"] Jul 13 17:28:47.816536 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.816514 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2btzw\" (UniqueName: \"kubernetes.io/projected/3def4529-53fd-4894-88d2-fea5def4b2d9-kube-api-access-2btzw\") pod \"kuadrant-operator-controller-manager-86878c769f-6f8d6\" (UID: \"3def4529-53fd-4894-88d2-fea5def4b2d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:47.816651 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.816569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3def4529-53fd-4894-88d2-fea5def4b2d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6f8d6\" (UID: \"3def4529-53fd-4894-88d2-fea5def4b2d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:47.917214 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.917180 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2btzw\" (UniqueName: \"kubernetes.io/projected/3def4529-53fd-4894-88d2-fea5def4b2d9-kube-api-access-2btzw\") pod \"kuadrant-operator-controller-manager-86878c769f-6f8d6\" (UID: \"3def4529-53fd-4894-88d2-fea5def4b2d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:47.917399 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.917279 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3def4529-53fd-4894-88d2-fea5def4b2d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6f8d6\" (UID: \"3def4529-53fd-4894-88d2-fea5def4b2d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:47.917752 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.917729 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3def4529-53fd-4894-88d2-fea5def4b2d9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6f8d6\" (UID: \"3def4529-53fd-4894-88d2-fea5def4b2d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:47.930095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.930072 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2btzw\" (UniqueName: \"kubernetes.io/projected/3def4529-53fd-4894-88d2-fea5def4b2d9-kube-api-access-2btzw\") pod \"kuadrant-operator-controller-manager-86878c769f-6f8d6\" (UID: \"3def4529-53fd-4894-88d2-fea5def4b2d9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:47.948334 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:47.948299 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:48.018183 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.018123 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93596795-3e4a-41a3-aeea-bfe300ad3b03-extensions-socket-volume\") pod \"93596795-3e4a-41a3-aeea-bfe300ad3b03\" (UID: \"93596795-3e4a-41a3-aeea-bfe300ad3b03\") " Jul 13 17:28:48.018183 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.018166 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkj4t\" (UniqueName: \"kubernetes.io/projected/93596795-3e4a-41a3-aeea-bfe300ad3b03-kube-api-access-xkj4t\") pod \"93596795-3e4a-41a3-aeea-bfe300ad3b03\" (UID: \"93596795-3e4a-41a3-aeea-bfe300ad3b03\") " Jul 13 17:28:48.018513 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.018488 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93596795-3e4a-41a3-aeea-bfe300ad3b03-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "93596795-3e4a-41a3-aeea-bfe300ad3b03" (UID: "93596795-3e4a-41a3-aeea-bfe300ad3b03"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:28:48.020115 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.020091 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93596795-3e4a-41a3-aeea-bfe300ad3b03-kube-api-access-xkj4t" (OuterVolumeSpecName: "kube-api-access-xkj4t") pod "93596795-3e4a-41a3-aeea-bfe300ad3b03" (UID: "93596795-3e4a-41a3-aeea-bfe300ad3b03"). InnerVolumeSpecName "kube-api-access-xkj4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:28:48.100936 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.100911 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:48.119287 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.119264 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/93596795-3e4a-41a3-aeea-bfe300ad3b03-extensions-socket-volume\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:28:48.119391 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.119288 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkj4t\" (UniqueName: \"kubernetes.io/projected/93596795-3e4a-41a3-aeea-bfe300ad3b03-kube-api-access-xkj4t\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:28:48.248088 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.248063 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6"] Jul 13 17:28:48.250176 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:28:48.250148 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3def4529_53fd_4894_88d2_fea5def4b2d9.slice/crio-a07a4e79109d9b1f6a7c3105903cac30d17f8e007cc635e37e5094b13e400459 WatchSource:0}: Error finding container a07a4e79109d9b1f6a7c3105903cac30d17f8e007cc635e37e5094b13e400459: Status 404 returned error can't find the container with id a07a4e79109d9b1f6a7c3105903cac30d17f8e007cc635e37e5094b13e400459 Jul 13 17:28:48.384618 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.384588 2579 generic.go:356] "Generic (PLEG): container finished" podID="93596795-3e4a-41a3-aeea-bfe300ad3b03" containerID="b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962" exitCode=0 Jul 13 17:28:48.384727 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.384629 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" Jul 13 17:28:48.384727 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.384670 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" event={"ID":"93596795-3e4a-41a3-aeea-bfe300ad3b03","Type":"ContainerDied","Data":"b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962"} Jul 13 17:28:48.384727 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.384723 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp" event={"ID":"93596795-3e4a-41a3-aeea-bfe300ad3b03","Type":"ContainerDied","Data":"3459a7fa6dc2e183df359d53dae7783ea93b8df9bde787a24983318620f4fe3d"} Jul 13 17:28:48.384849 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.384753 2579 scope.go:117] "RemoveContainer" containerID="b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962" Jul 13 17:28:48.386359 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.386338 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" event={"ID":"3def4529-53fd-4894-88d2-fea5def4b2d9","Type":"ContainerStarted","Data":"f9bcb0bce415ae172e7fdc9dd3dff9154521fb9e0bf1c3f583f99e86be45ca88"} Jul 13 17:28:48.386459 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.386382 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:28:48.386459 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.386401 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" event={"ID":"3def4529-53fd-4894-88d2-fea5def4b2d9","Type":"ContainerStarted","Data":"a07a4e79109d9b1f6a7c3105903cac30d17f8e007cc635e37e5094b13e400459"} Jul 13 17:28:48.393678 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.393658 2579 scope.go:117] "RemoveContainer" containerID="b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962" Jul 13 17:28:48.414236 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:28:48.414206 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962\": container with ID starting with b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962 not found: ID does not exist" containerID="b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962" Jul 13 17:28:48.414385 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.414246 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962"} err="failed to get container status \"b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962\": rpc error: code = NotFound desc = could not find container \"b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962\": container with ID starting with b73d44c322537b2380a2985bc08643289ffb5db9dc5c36a57e590903a5d63962 not found: ID does not exist" Jul 13 17:28:48.416217 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.416161 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" podStartSLOduration=1.416144117 podStartE2EDuration="1.416144117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:28:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:28:48.413983625 +0000 UTC m=+666.570470502" watchObservedRunningTime="2026-07-13 17:28:48.416144117 +0000 UTC m=+666.572630986" Jul 13 17:28:48.438872 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.436107 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp"] Jul 13 17:28:48.443929 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.443906 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9fnrp"] Jul 13 17:28:48.661660 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:48.661622 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93596795-3e4a-41a3-aeea-bfe300ad3b03" path="/var/lib/kubelet/pods/93596795-3e4a-41a3-aeea-bfe300ad3b03/volumes" Jul 13 17:28:58.221990 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:58.221948 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b7dbbd65f-zfqb2"] Jul 13 17:28:59.394905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:28:59.394871 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:29:03.915256 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:03.915223 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7"] Jul 13 17:29:03.915825 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:03.915801 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93596795-3e4a-41a3-aeea-bfe300ad3b03" containerName="manager" Jul 13 17:29:03.915825 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:03.915825 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="93596795-3e4a-41a3-aeea-bfe300ad3b03" containerName="manager" Jul 13 17:29:03.916011 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:03.915947 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="93596795-3e4a-41a3-aeea-bfe300ad3b03" containerName="manager" Jul 13 17:29:03.918926 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:03.918908 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:03.921574 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:03.921554 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-zwdc8\"" Jul 13 17:29:03.934669 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:03.934640 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7"] Jul 13 17:29:04.041474 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041446 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.041591 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041503 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.041635 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041584 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f348f376-d3b1-4b27-b342-236e5b92e9a3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.041635 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041620 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.041701 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041644 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.041734 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041700 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.041734 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041727 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.041794 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dghc\" (UniqueName: \"kubernetes.io/projected/f348f376-d3b1-4b27-b342-236e5b92e9a3-kube-api-access-4dghc\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.041794 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.041764 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.142560 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f348f376-d3b1-4b27-b342-236e5b92e9a3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.142681 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142576 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.142681 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142601 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.142681 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142624 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.142681 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142638 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.142681 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142654 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dghc\" (UniqueName: \"kubernetes.io/projected/f348f376-d3b1-4b27-b342-236e5b92e9a3-kube-api-access-4dghc\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.142930 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142699 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.142930 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142795 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.143057 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142953 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.143057 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.142969 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.143164 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.143143 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.143218 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.143175 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.143408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.143383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.143507 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.143485 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f348f376-d3b1-4b27-b342-236e5b92e9a3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.145379 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.145356 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.145617 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.145593 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.154126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.154071 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f348f376-d3b1-4b27-b342-236e5b92e9a3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.154126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.154093 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dghc\" (UniqueName: \"kubernetes.io/projected/f348f376-d3b1-4b27-b342-236e5b92e9a3-kube-api-access-4dghc\") pod \"maas-default-gateway-openshift-default-56654dcf7-rk8z7\" (UID: \"f348f376-d3b1-4b27-b342-236e5b92e9a3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.229045 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.228991 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:04.362092 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.362067 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7"] Jul 13 17:29:04.363315 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:29:04.363284 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf348f376_d3b1_4b27_b342_236e5b92e9a3.slice/crio-07564019f2764e48d4893390bb3ff261ab3a668e6fbedc1549b88a68b61539e1 WatchSource:0}: Error finding container 07564019f2764e48d4893390bb3ff261ab3a668e6fbedc1549b88a68b61539e1: Status 404 returned error can't find the container with id 07564019f2764e48d4893390bb3ff261ab3a668e6fbedc1549b88a68b61539e1 Jul 13 17:29:04.365452 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.365418 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:29:04.365536 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.365488 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:29:04.365536 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.365517 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:29:04.475994 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.475956 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" event={"ID":"f348f376-d3b1-4b27-b342-236e5b92e9a3","Type":"ContainerStarted","Data":"45808f66548dac79dac0d2aad960cacf622b36fde96c7a9f033ebf3e2f1775dd"} Jul 13 17:29:04.476149 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.476023 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" event={"ID":"f348f376-d3b1-4b27-b342-236e5b92e9a3","Type":"ContainerStarted","Data":"07564019f2764e48d4893390bb3ff261ab3a668e6fbedc1549b88a68b61539e1"} Jul 13 17:29:04.502016 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:04.501917 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" podStartSLOduration=1.501896634 podStartE2EDuration="1.501896634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:29:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:29:04.497665808 +0000 UTC m=+682.654152678" watchObservedRunningTime="2026-07-13 17:29:04.501896634 +0000 UTC m=+682.658383506" Jul 13 17:29:05.229147 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:05.229116 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:05.234055 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:05.234028 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:05.481991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:05.481927 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:05.482824 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:05.482806 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:29:23.242728 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.242663 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b7dbbd65f-zfqb2" podUID="11ba2824-84f5-46d4-a602-3b1d349639b9" containerName="console" containerID="cri-o://42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246" gracePeriod=15 Jul 13 17:29:23.510246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.510214 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:29:23.579882 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.579843 2579 generic.go:356] "Generic (PLEG): container finished" podID="11ba2824-84f5-46d4-a602-3b1d349639b9" containerID="42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246" exitCode=0 Jul 13 17:29:23.580070 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.579899 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7dbbd65f-zfqb2" Jul 13 17:29:23.580070 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.579932 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7dbbd65f-zfqb2" event={"ID":"11ba2824-84f5-46d4-a602-3b1d349639b9","Type":"ContainerDied","Data":"42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246"} Jul 13 17:29:23.580070 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.579988 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7dbbd65f-zfqb2" event={"ID":"11ba2824-84f5-46d4-a602-3b1d349639b9","Type":"ContainerDied","Data":"f666a2077782c7aa4026f5f4a1e9ba2ba1fc23988e0afdef3ff207f743da3939"} Jul 13 17:29:23.580070 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.580012 2579 scope.go:117] "RemoveContainer" containerID="42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246" Jul 13 17:29:23.589021 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.588999 2579 scope.go:117] "RemoveContainer" containerID="42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246" Jul 13 17:29:23.589352 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:29:23.589304 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246\": container with ID starting with 42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246 not found: ID does not exist" containerID="42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246" Jul 13 17:29:23.589423 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.589364 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246"} err="failed to get container status \"42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246\": rpc error: code = NotFound desc = could not find container \"42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246\": container with ID starting with 42f6d1b08e0aed52d82781fd45180c4b913186028e192a5b858f91f1abec5246 not found: ID does not exist" Jul 13 17:29:23.619831 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.619797 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-trusted-ca-bundle\") pod \"11ba2824-84f5-46d4-a602-3b1d349639b9\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " Jul 13 17:29:23.619946 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.619865 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-oauth-config\") pod \"11ba2824-84f5-46d4-a602-3b1d349639b9\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " Jul 13 17:29:23.619946 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.619896 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lq5t7\" (UniqueName: \"kubernetes.io/projected/11ba2824-84f5-46d4-a602-3b1d349639b9-kube-api-access-lq5t7\") pod \"11ba2824-84f5-46d4-a602-3b1d349639b9\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " Jul 13 17:29:23.619946 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.619913 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-serving-cert\") pod \"11ba2824-84f5-46d4-a602-3b1d349639b9\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " Jul 13 17:29:23.619946 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.619941 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-service-ca\") pod \"11ba2824-84f5-46d4-a602-3b1d349639b9\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " Jul 13 17:29:23.620147 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.619962 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-oauth-serving-cert\") pod \"11ba2824-84f5-46d4-a602-3b1d349639b9\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " Jul 13 17:29:23.620147 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.620046 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-console-config\") pod \"11ba2824-84f5-46d4-a602-3b1d349639b9\" (UID: \"11ba2824-84f5-46d4-a602-3b1d349639b9\") " Jul 13 17:29:23.620555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.620406 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "11ba2824-84f5-46d4-a602-3b1d349639b9" (UID: "11ba2824-84f5-46d4-a602-3b1d349639b9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:29:23.620691 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.620566 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "11ba2824-84f5-46d4-a602-3b1d349639b9" (UID: "11ba2824-84f5-46d4-a602-3b1d349639b9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:29:23.620691 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.620577 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-service-ca" (OuterVolumeSpecName: "service-ca") pod "11ba2824-84f5-46d4-a602-3b1d349639b9" (UID: "11ba2824-84f5-46d4-a602-3b1d349639b9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:29:23.620691 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.620646 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-console-config" (OuterVolumeSpecName: "console-config") pod "11ba2824-84f5-46d4-a602-3b1d349639b9" (UID: "11ba2824-84f5-46d4-a602-3b1d349639b9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:29:23.622537 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.622506 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11ba2824-84f5-46d4-a602-3b1d349639b9-kube-api-access-lq5t7" (OuterVolumeSpecName: "kube-api-access-lq5t7") pod "11ba2824-84f5-46d4-a602-3b1d349639b9" (UID: "11ba2824-84f5-46d4-a602-3b1d349639b9"). InnerVolumeSpecName "kube-api-access-lq5t7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:29:23.622720 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.622691 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "11ba2824-84f5-46d4-a602-3b1d349639b9" (UID: "11ba2824-84f5-46d4-a602-3b1d349639b9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:29:23.622812 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.622722 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "11ba2824-84f5-46d4-a602-3b1d349639b9" (UID: "11ba2824-84f5-46d4-a602-3b1d349639b9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:29:23.721341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.721155 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-trusted-ca-bundle\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:29:23.721341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.721194 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-oauth-config\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:29:23.721341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.721210 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq5t7\" (UniqueName: \"kubernetes.io/projected/11ba2824-84f5-46d4-a602-3b1d349639b9-kube-api-access-lq5t7\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:29:23.721341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.721224 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11ba2824-84f5-46d4-a602-3b1d349639b9-console-serving-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:29:23.721341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.721237 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-service-ca\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:29:23.721341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.721253 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-oauth-serving-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:29:23.721341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.721267 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11ba2824-84f5-46d4-a602-3b1d349639b9-console-config\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:29:23.903463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.903415 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b7dbbd65f-zfqb2"] Jul 13 17:29:23.910198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:23.910148 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b7dbbd65f-zfqb2"] Jul 13 17:29:24.664560 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:24.664524 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11ba2824-84f5-46d4-a602-3b1d349639b9" path="/var/lib/kubelet/pods/11ba2824-84f5-46d4-a602-3b1d349639b9/volumes" Jul 13 17:29:49.360985 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.360940 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-646df8d49-v9st7"] Jul 13 17:29:49.362450 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.362416 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ba2824-84f5-46d4-a602-3b1d349639b9" containerName="console" Jul 13 17:29:49.362640 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.362627 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ba2824-84f5-46d4-a602-3b1d349639b9" containerName="console" Jul 13 17:29:49.362967 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.362953 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="11ba2824-84f5-46d4-a602-3b1d349639b9" containerName="console" Jul 13 17:29:49.368995 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.368957 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:49.373115 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.373087 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 17:29:49.373235 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.373191 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 17:29:49.373235 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.373200 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vv9zn\"" Jul 13 17:29:49.379056 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.379026 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-646df8d49-v9st7"] Jul 13 17:29:49.461021 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.460974 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px2kg\" (UniqueName: \"kubernetes.io/projected/a988f87c-ff07-4804-9101-9aea0d78149c-kube-api-access-px2kg\") pod \"maas-controller-646df8d49-v9st7\" (UID: \"a988f87c-ff07-4804-9101-9aea0d78149c\") " pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:49.461209 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.461040 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a988f87c-ff07-4804-9101-9aea0d78149c-webhook-cert\") pod \"maas-controller-646df8d49-v9st7\" (UID: \"a988f87c-ff07-4804-9101-9aea0d78149c\") " pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:49.561844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.561808 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px2kg\" (UniqueName: \"kubernetes.io/projected/a988f87c-ff07-4804-9101-9aea0d78149c-kube-api-access-px2kg\") pod \"maas-controller-646df8d49-v9st7\" (UID: \"a988f87c-ff07-4804-9101-9aea0d78149c\") " pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:49.562052 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.561868 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a988f87c-ff07-4804-9101-9aea0d78149c-webhook-cert\") pod \"maas-controller-646df8d49-v9st7\" (UID: \"a988f87c-ff07-4804-9101-9aea0d78149c\") " pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:49.564392 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.564364 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a988f87c-ff07-4804-9101-9aea0d78149c-webhook-cert\") pod \"maas-controller-646df8d49-v9st7\" (UID: \"a988f87c-ff07-4804-9101-9aea0d78149c\") " pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:49.572125 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.572096 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px2kg\" (UniqueName: \"kubernetes.io/projected/a988f87c-ff07-4804-9101-9aea0d78149c-kube-api-access-px2kg\") pod \"maas-controller-646df8d49-v9st7\" (UID: \"a988f87c-ff07-4804-9101-9aea0d78149c\") " pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:49.681193 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.681151 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:49.842386 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:49.842354 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-646df8d49-v9st7"] Jul 13 17:29:49.844906 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:29:49.844876 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda988f87c_ff07_4804_9101_9aea0d78149c.slice/crio-d7bd15738a5bcee7567c179729661ffd9324302d65c089ee85886c77b25fddd6 WatchSource:0}: Error finding container d7bd15738a5bcee7567c179729661ffd9324302d65c089ee85886c77b25fddd6: Status 404 returned error can't find the container with id d7bd15738a5bcee7567c179729661ffd9324302d65c089ee85886c77b25fddd6 Jul 13 17:29:50.730681 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:50.730601 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-646df8d49-v9st7" event={"ID":"a988f87c-ff07-4804-9101-9aea0d78149c","Type":"ContainerStarted","Data":"d7bd15738a5bcee7567c179729661ffd9324302d65c089ee85886c77b25fddd6"} Jul 13 17:29:52.747362 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:52.747296 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-646df8d49-v9st7" event={"ID":"a988f87c-ff07-4804-9101-9aea0d78149c","Type":"ContainerStarted","Data":"04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4"} Jul 13 17:29:52.747834 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:52.747382 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:29:52.770088 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:29:52.770027 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-646df8d49-v9st7" podStartSLOduration=1.5416772060000001 podStartE2EDuration="3.770010146s" totalImagesPullingTime="2.22833294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:29:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:29:52.76741321 +0000 UTC m=+730.923900084" watchObservedRunningTime="2026-07-13 17:29:52.770010146 +0000 UTC m=+730.926497021" Jul 13 17:30:03.755973 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:03.755938 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:30:04.492132 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.492088 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-646df8d49-v9st7"] Jul 13 17:30:04.492394 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.492343 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-646df8d49-v9st7" podUID="a988f87c-ff07-4804-9101-9aea0d78149c" containerName="manager" containerID="cri-o://04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4" gracePeriod=30 Jul 13 17:30:04.745636 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.745565 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:30:04.801843 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.801801 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px2kg\" (UniqueName: \"kubernetes.io/projected/a988f87c-ff07-4804-9101-9aea0d78149c-kube-api-access-px2kg\") pod \"a988f87c-ff07-4804-9101-9aea0d78149c\" (UID: \"a988f87c-ff07-4804-9101-9aea0d78149c\") " Jul 13 17:30:04.802229 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.801930 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a988f87c-ff07-4804-9101-9aea0d78149c-webhook-cert\") pod \"a988f87c-ff07-4804-9101-9aea0d78149c\" (UID: \"a988f87c-ff07-4804-9101-9aea0d78149c\") " Jul 13 17:30:04.804121 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.804085 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a988f87c-ff07-4804-9101-9aea0d78149c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a988f87c-ff07-4804-9101-9aea0d78149c" (UID: "a988f87c-ff07-4804-9101-9aea0d78149c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 17:30:04.804262 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.804211 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a988f87c-ff07-4804-9101-9aea0d78149c-kube-api-access-px2kg" (OuterVolumeSpecName: "kube-api-access-px2kg") pod "a988f87c-ff07-4804-9101-9aea0d78149c" (UID: "a988f87c-ff07-4804-9101-9aea0d78149c"). InnerVolumeSpecName "kube-api-access-px2kg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:30:04.813620 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.813593 2579 generic.go:356] "Generic (PLEG): container finished" podID="a988f87c-ff07-4804-9101-9aea0d78149c" containerID="04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4" exitCode=0 Jul 13 17:30:04.813731 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.813638 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-646df8d49-v9st7" Jul 13 17:30:04.813731 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.813672 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-646df8d49-v9st7" event={"ID":"a988f87c-ff07-4804-9101-9aea0d78149c","Type":"ContainerDied","Data":"04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4"} Jul 13 17:30:04.813809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.813730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-646df8d49-v9st7" event={"ID":"a988f87c-ff07-4804-9101-9aea0d78149c","Type":"ContainerDied","Data":"d7bd15738a5bcee7567c179729661ffd9324302d65c089ee85886c77b25fddd6"} Jul 13 17:30:04.813809 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.813757 2579 scope.go:117] "RemoveContainer" containerID="04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4" Jul 13 17:30:04.824000 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.823974 2579 scope.go:117] "RemoveContainer" containerID="04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4" Jul 13 17:30:04.824425 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:30:04.824398 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4\": container with ID starting with 04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4 not found: ID does not exist" containerID="04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4" Jul 13 17:30:04.824602 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.824439 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4"} err="failed to get container status \"04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4\": rpc error: code = NotFound desc = could not find container \"04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4\": container with ID starting with 04e097e5878dae89bcab621d3310c70b80bb7429dec8eb014e1a3ffd266452d4 not found: ID does not exist" Jul 13 17:30:04.838091 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.838057 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-646df8d49-v9st7"] Jul 13 17:30:04.846677 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.846636 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-646df8d49-v9st7"] Jul 13 17:30:04.903296 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.903261 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px2kg\" (UniqueName: \"kubernetes.io/projected/a988f87c-ff07-4804-9101-9aea0d78149c-kube-api-access-px2kg\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:30:04.903296 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:04.903291 2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a988f87c-ff07-4804-9101-9aea0d78149c-webhook-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:30:06.668242 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:06.668196 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a988f87c-ff07-4804-9101-9aea0d78149c" path="/var/lib/kubelet/pods/a988f87c-ff07-4804-9101-9aea0d78149c/volumes" Jul 13 17:30:25.148295 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.148256 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-868b79db4-wrtlq"] Jul 13 17:30:25.148905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.148884 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a988f87c-ff07-4804-9101-9aea0d78149c" containerName="manager" Jul 13 17:30:25.148905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.148906 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a988f87c-ff07-4804-9101-9aea0d78149c" containerName="manager" Jul 13 17:30:25.149073 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.148969 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a988f87c-ff07-4804-9101-9aea0d78149c" containerName="manager" Jul 13 17:30:25.155434 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.155411 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.157845 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.157822 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 17:30:25.157977 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.157845 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 17:30:25.158534 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.158516 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-qt2bw\"" Jul 13 17:30:25.158625 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.158520 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 17:30:25.163838 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.163816 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-868b79db4-wrtlq"] Jul 13 17:30:25.286266 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.286218 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-466w9\" (UniqueName: \"kubernetes.io/projected/1747b87b-bebe-44fd-9d19-a257ef47b2dd-kube-api-access-466w9\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.286504 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.286314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1747b87b-bebe-44fd-9d19-a257ef47b2dd-maas-api-tls\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.286504 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.286382 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1747b87b-bebe-44fd-9d19-a257ef47b2dd-openshift-service-ca\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.387432 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.387391 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1747b87b-bebe-44fd-9d19-a257ef47b2dd-maas-api-tls\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.387819 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.387795 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1747b87b-bebe-44fd-9d19-a257ef47b2dd-openshift-service-ca\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.388041 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.388021 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-466w9\" (UniqueName: \"kubernetes.io/projected/1747b87b-bebe-44fd-9d19-a257ef47b2dd-kube-api-access-466w9\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.388772 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.388747 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1747b87b-bebe-44fd-9d19-a257ef47b2dd-openshift-service-ca\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.391204 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.391176 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1747b87b-bebe-44fd-9d19-a257ef47b2dd-maas-api-tls\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.397478 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.397455 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-466w9\" (UniqueName: \"kubernetes.io/projected/1747b87b-bebe-44fd-9d19-a257ef47b2dd-kube-api-access-466w9\") pod \"maas-api-868b79db4-wrtlq\" (UID: \"1747b87b-bebe-44fd-9d19-a257ef47b2dd\") " pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.467392 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.467278 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:25.624047 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.623994 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-868b79db4-wrtlq"] Jul 13 17:30:25.625637 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:30:25.625604 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1747b87b_bebe_44fd_9d19_a257ef47b2dd.slice/crio-d6b573d67ca57fae6d72b5c1c7927fc76a271d6de5be74e154785eaae962f04e WatchSource:0}: Error finding container d6b573d67ca57fae6d72b5c1c7927fc76a271d6de5be74e154785eaae962f04e: Status 404 returned error can't find the container with id d6b573d67ca57fae6d72b5c1c7927fc76a271d6de5be74e154785eaae962f04e Jul 13 17:30:25.627051 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.627032 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:30:25.928203 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:25.928161 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" event={"ID":"1747b87b-bebe-44fd-9d19-a257ef47b2dd","Type":"ContainerStarted","Data":"d6b573d67ca57fae6d72b5c1c7927fc76a271d6de5be74e154785eaae962f04e"} Jul 13 17:30:28.967557 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:28.967513 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" event={"ID":"1747b87b-bebe-44fd-9d19-a257ef47b2dd","Type":"ContainerStarted","Data":"f17d4186d9eca73ddce9782ad2aa746f8258574ca321ea6012f957409114134d"} Jul 13 17:30:28.967988 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:28.967585 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:28.990082 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:28.990014 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" podStartSLOduration=1.313644883 podStartE2EDuration="3.989995032s" totalImagesPullingTime="2.676350149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:30:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:30:28.989924478 +0000 UTC m=+767.146411350" watchObservedRunningTime="2026-07-13 17:30:28.989995032 +0000 UTC m=+767.146481907" Jul 13 17:30:34.004255 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:34.004220 2579 generic.go:356] "Generic (PLEG): container finished" podID="f348f376-d3b1-4b27-b342-236e5b92e9a3" containerID="45808f66548dac79dac0d2aad960cacf622b36fde96c7a9f033ebf3e2f1775dd" exitCode=0 Jul 13 17:30:34.004737 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:34.004295 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" event={"ID":"f348f376-d3b1-4b27-b342-236e5b92e9a3","Type":"ContainerDied","Data":"45808f66548dac79dac0d2aad960cacf622b36fde96c7a9f033ebf3e2f1775dd"} Jul 13 17:30:34.004783 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:34.004745 2579 scope.go:117] "RemoveContainer" containerID="45808f66548dac79dac0d2aad960cacf622b36fde96c7a9f033ebf3e2f1775dd" Jul 13 17:30:34.005835 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:34.005798 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:30:34.005911 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:34.005892 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:30:34.005969 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:34.005951 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:30:34.979465 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:34.979433 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-868b79db4-wrtlq" Jul 13 17:30:35.014010 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:35.013976 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" event={"ID":"f348f376-d3b1-4b27-b342-236e5b92e9a3","Type":"ContainerStarted","Data":"ab5ada9f3278485504e32771d194ee4c1e9537e7737fa08aa1449285196453f1"} Jul 13 17:30:35.229238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:35.229191 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:30:35.231020 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:35.230976 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rk8z7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 13 17:30:35.231182 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:35.231045 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" podUID="f348f376-d3b1-4b27-b342-236e5b92e9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 13 17:30:35.482451 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:35.482353 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:30:36.229943 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:36.229900 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rk8z7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 13 17:30:36.230503 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:36.230007 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" podUID="f348f376-d3b1-4b27-b342-236e5b92e9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 13 17:30:37.229714 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:37.229665 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rk8z7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 13 17:30:37.229933 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:37.229751 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" podUID="f348f376-d3b1-4b27-b342-236e5b92e9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 13 17:30:38.230353 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:38.230296 2579 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rk8z7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 13 17:30:38.231542 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:38.231080 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" podUID="f348f376-d3b1-4b27-b342-236e5b92e9a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 13 17:30:39.234181 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:39.234148 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:30:39.235258 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:39.235235 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rk8z7" Jul 13 17:30:48.868137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.868098 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm"] Jul 13 17:30:48.873250 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.873213 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:48.875597 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.875574 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 13 17:30:48.875597 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.875574 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 17:30:48.875911 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.875892 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 17:30:48.876453 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.876434 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-srn9x\"" Jul 13 17:30:48.886558 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.886536 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm"] Jul 13 17:30:48.917929 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.917898 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:48.918119 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.917936 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkjqp\" (UniqueName: \"kubernetes.io/projected/3f6400a4-644a-4d31-8570-b2e58eb70925-kube-api-access-kkjqp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:48.918119 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.917969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:48.918213 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.918117 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:48.918213 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.918163 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f6400a4-644a-4d31-8570-b2e58eb70925-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:48.918213 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:48.918189 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.019619 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.019577 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f6400a4-644a-4d31-8570-b2e58eb70925-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.019619 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.019623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.019875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.019664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.019875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.019688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkjqp\" (UniqueName: \"kubernetes.io/projected/3f6400a4-644a-4d31-8570-b2e58eb70925-kube-api-access-kkjqp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.019875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.019715 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.019875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.019785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.020138 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.020109 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.020265 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.020126 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.020265 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.020156 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.022143 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.022119 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f6400a4-644a-4d31-8570-b2e58eb70925-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.022538 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.022517 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f6400a4-644a-4d31-8570-b2e58eb70925-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.028745 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.028717 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkjqp\" (UniqueName: \"kubernetes.io/projected/3f6400a4-644a-4d31-8570-b2e58eb70925-kube-api-access-kkjqp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-jsxtm\" (UID: \"3f6400a4-644a-4d31-8570-b2e58eb70925\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.185019 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.184969 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:49.333105 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:49.333069 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm"] Jul 13 17:30:49.335075 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:30:49.335044 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6400a4_644a_4d31_8570_b2e58eb70925.slice/crio-9a269c863f395f2d6fe6fc1f4b946879fa4c83b41a7fee1b4e15b89249ff283e WatchSource:0}: Error finding container 9a269c863f395f2d6fe6fc1f4b946879fa4c83b41a7fee1b4e15b89249ff283e: Status 404 returned error can't find the container with id 9a269c863f395f2d6fe6fc1f4b946879fa4c83b41a7fee1b4e15b89249ff283e Jul 13 17:30:50.108568 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:50.108528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" event={"ID":"3f6400a4-644a-4d31-8570-b2e58eb70925","Type":"ContainerStarted","Data":"9a269c863f395f2d6fe6fc1f4b946879fa4c83b41a7fee1b4e15b89249ff283e"} Jul 13 17:30:51.116349 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:51.116287 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" event={"ID":"3f6400a4-644a-4d31-8570-b2e58eb70925","Type":"ContainerStarted","Data":"9164e787602508948ed3fa6d2cd4a9783712222c26b6fc6328429495def480fd"} Jul 13 17:30:51.133166 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:51.133102 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" podStartSLOduration=2.057188121 podStartE2EDuration="3.133082235s" totalImagesPullingTime="1.075894114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:30:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:30:51.132766843 +0000 UTC m=+789.289253719" watchObservedRunningTime="2026-07-13 17:30:51.133082235 +0000 UTC m=+789.289569111" Jul 13 17:30:59.185702 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:59.185653 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:59.186194 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:59.185726 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:30:59.198833 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:30:59.198795 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:31:00.179211 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:00.179180 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-jsxtm" Jul 13 17:31:14.170623 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.170525 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng"] Jul 13 17:31:14.175257 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.175225 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.177588 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.177563 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 17:31:14.189039 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.189011 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng"] Jul 13 17:31:14.265678 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.265643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.265875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.265698 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.265875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.265772 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.265875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.265808 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc6ms\" (UniqueName: \"kubernetes.io/projected/e1adfb80-fc2c-4c56-924d-a2824f8a013a-kube-api-access-kc6ms\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.265875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.265843 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.265875 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.265859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1adfb80-fc2c-4c56-924d-a2824f8a013a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367011 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.366966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367011 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.367011 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1adfb80-fc2c-4c56-924d-a2824f8a013a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367249 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.367074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367249 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.367215 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367350 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.367278 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367350 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.367305 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc6ms\" (UniqueName: \"kubernetes.io/projected/e1adfb80-fc2c-4c56-924d-a2824f8a013a-kube-api-access-kc6ms\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367543 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.367522 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367628 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.367558 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.367689 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.367668 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.369530 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.369509 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1adfb80-fc2c-4c56-924d-a2824f8a013a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.369670 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.369654 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1adfb80-fc2c-4c56-924d-a2824f8a013a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.376903 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.376882 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc6ms\" (UniqueName: \"kubernetes.io/projected/e1adfb80-fc2c-4c56-924d-a2824f8a013a-kube-api-access-kc6ms\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng\" (UID: \"e1adfb80-fc2c-4c56-924d-a2824f8a013a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.487750 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.487652 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:14.637613 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:14.637584 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng"] Jul 13 17:31:14.640388 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:31:14.640354 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1adfb80_fc2c_4c56_924d_a2824f8a013a.slice/crio-a1be20bd7fc42f9573cea022f3b4cfa24f9032fb9334029892829b1e4a8553ed WatchSource:0}: Error finding container a1be20bd7fc42f9573cea022f3b4cfa24f9032fb9334029892829b1e4a8553ed: Status 404 returned error can't find the container with id a1be20bd7fc42f9573cea022f3b4cfa24f9032fb9334029892829b1e4a8553ed Jul 13 17:31:15.259689 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:15.259645 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" event={"ID":"e1adfb80-fc2c-4c56-924d-a2824f8a013a","Type":"ContainerStarted","Data":"93820ba9ae1087a7fe98a01ec691ec31d6ff5f160d1648ad22735cf078592169"} Jul 13 17:31:15.260201 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:15.259753 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" event={"ID":"e1adfb80-fc2c-4c56-924d-a2824f8a013a","Type":"ContainerStarted","Data":"a1be20bd7fc42f9573cea022f3b4cfa24f9032fb9334029892829b1e4a8553ed"} Jul 13 17:31:15.298637 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:15.298567 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" podStartSLOduration=1.092890361 podStartE2EDuration="1.298547286s" totalImagesPullingTime="205.656925ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:31:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:31:15.29500939 +0000 UTC m=+813.451496265" watchObservedRunningTime="2026-07-13 17:31:15.298547286 +0000 UTC m=+813.455034162" Jul 13 17:31:24.488509 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:24.488461 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:24.488936 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:24.488548 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:24.501173 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:24.501139 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:31:25.333107 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:31:25.333072 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng" Jul 13 17:32:42.531989 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:32:42.531951 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:32:42.536151 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:32:42.536107 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:37:42.570707 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:37:42.570673 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:37:42.574645 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:37:42.574624 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:38:33.417743 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:33.417709 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6"] Jul 13 17:38:33.418289 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:33.417900 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" podUID="3def4529-53fd-4894-88d2-fea5def4b2d9" containerName="manager" containerID="cri-o://f9bcb0bce415ae172e7fdc9dd3dff9154521fb9e0bf1c3f583f99e86be45ca88" gracePeriod=10 Jul 13 17:38:36.733244 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:36.733156 2579 generic.go:356] "Generic (PLEG): container finished" podID="3def4529-53fd-4894-88d2-fea5def4b2d9" containerID="f9bcb0bce415ae172e7fdc9dd3dff9154521fb9e0bf1c3f583f99e86be45ca88" exitCode=0 Jul 13 17:38:36.733244 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:36.733206 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" event={"ID":"3def4529-53fd-4894-88d2-fea5def4b2d9","Type":"ContainerDied","Data":"f9bcb0bce415ae172e7fdc9dd3dff9154521fb9e0bf1c3f583f99e86be45ca88"} Jul 13 17:38:37.070218 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.070190 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:38:37.149448 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.149415 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3def4529-53fd-4894-88d2-fea5def4b2d9-extensions-socket-volume\") pod \"3def4529-53fd-4894-88d2-fea5def4b2d9\" (UID: \"3def4529-53fd-4894-88d2-fea5def4b2d9\") " Jul 13 17:38:37.149629 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.149518 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2btzw\" (UniqueName: \"kubernetes.io/projected/3def4529-53fd-4894-88d2-fea5def4b2d9-kube-api-access-2btzw\") pod \"3def4529-53fd-4894-88d2-fea5def4b2d9\" (UID: \"3def4529-53fd-4894-88d2-fea5def4b2d9\") " Jul 13 17:38:37.149886 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.149859 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3def4529-53fd-4894-88d2-fea5def4b2d9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3def4529-53fd-4894-88d2-fea5def4b2d9" (UID: "3def4529-53fd-4894-88d2-fea5def4b2d9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:38:37.151599 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.151576 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3def4529-53fd-4894-88d2-fea5def4b2d9-kube-api-access-2btzw" (OuterVolumeSpecName: "kube-api-access-2btzw") pod "3def4529-53fd-4894-88d2-fea5def4b2d9" (UID: "3def4529-53fd-4894-88d2-fea5def4b2d9"). InnerVolumeSpecName "kube-api-access-2btzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:38:37.250726 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.250642 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2btzw\" (UniqueName: \"kubernetes.io/projected/3def4529-53fd-4894-88d2-fea5def4b2d9-kube-api-access-2btzw\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:38:37.250726 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.250672 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3def4529-53fd-4894-88d2-fea5def4b2d9-extensions-socket-volume\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:38:37.740449 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.740415 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" Jul 13 17:38:37.740855 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.740421 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6" event={"ID":"3def4529-53fd-4894-88d2-fea5def4b2d9","Type":"ContainerDied","Data":"a07a4e79109d9b1f6a7c3105903cac30d17f8e007cc635e37e5094b13e400459"} Jul 13 17:38:37.740855 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.740591 2579 scope.go:117] "RemoveContainer" containerID="f9bcb0bce415ae172e7fdc9dd3dff9154521fb9e0bf1c3f583f99e86be45ca88" Jul 13 17:38:37.764744 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.764716 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6"] Jul 13 17:38:37.768482 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:37.768459 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6f8d6"] Jul 13 17:38:38.661228 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:38:38.661199 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3def4529-53fd-4894-88d2-fea5def4b2d9" path="/var/lib/kubelet/pods/3def4529-53fd-4894-88d2-fea5def4b2d9/volumes" Jul 13 17:39:34.495840 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.495805 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd"] Jul 13 17:39:34.496313 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.496175 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3def4529-53fd-4894-88d2-fea5def4b2d9" containerName="manager" Jul 13 17:39:34.496313 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.496186 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3def4529-53fd-4894-88d2-fea5def4b2d9" containerName="manager" Jul 13 17:39:34.496313 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.496258 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3def4529-53fd-4894-88d2-fea5def4b2d9" containerName="manager" Jul 13 17:39:34.499296 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.499275 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:34.503234 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.503213 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 17:39:34.503364 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.503297 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t6clk\"" Jul 13 17:39:34.503667 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.503648 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 17:39:34.522254 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.522224 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd"] Jul 13 17:39:34.537671 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.537643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e334d773-74b6-41d4-b1bc-d0c0e4fb8f81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q2djd\" (UID: \"e334d773-74b6-41d4-b1bc-d0c0e4fb8f81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:34.537787 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.537701 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czjt6\" (UniqueName: \"kubernetes.io/projected/e334d773-74b6-41d4-b1bc-d0c0e4fb8f81-kube-api-access-czjt6\") pod \"kuadrant-operator-controller-manager-86878c769f-q2djd\" (UID: \"e334d773-74b6-41d4-b1bc-d0c0e4fb8f81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:34.638597 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.638567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjt6\" (UniqueName: \"kubernetes.io/projected/e334d773-74b6-41d4-b1bc-d0c0e4fb8f81-kube-api-access-czjt6\") pod \"kuadrant-operator-controller-manager-86878c769f-q2djd\" (UID: \"e334d773-74b6-41d4-b1bc-d0c0e4fb8f81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:34.638713 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.638694 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e334d773-74b6-41d4-b1bc-d0c0e4fb8f81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q2djd\" (UID: \"e334d773-74b6-41d4-b1bc-d0c0e4fb8f81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:34.639058 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.639039 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e334d773-74b6-41d4-b1bc-d0c0e4fb8f81-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-q2djd\" (UID: \"e334d773-74b6-41d4-b1bc-d0c0e4fb8f81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:34.648041 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.648011 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czjt6\" (UniqueName: \"kubernetes.io/projected/e334d773-74b6-41d4-b1bc-d0c0e4fb8f81-kube-api-access-czjt6\") pod \"kuadrant-operator-controller-manager-86878c769f-q2djd\" (UID: \"e334d773-74b6-41d4-b1bc-d0c0e4fb8f81\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:34.809790 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.809702 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:34.966078 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:34.966048 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd"] Jul 13 17:39:34.968187 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:39:34.968157 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode334d773_74b6_41d4_b1bc_d0c0e4fb8f81.slice/crio-e436950d3c5f68eb269847a0ff2306bf0f61f98cd6fa72d5abf2841795e5ac8f WatchSource:0}: Error finding container e436950d3c5f68eb269847a0ff2306bf0f61f98cd6fa72d5abf2841795e5ac8f: Status 404 returned error can't find the container with id e436950d3c5f68eb269847a0ff2306bf0f61f98cd6fa72d5abf2841795e5ac8f Jul 13 17:39:35.046762 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:35.046715 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" event={"ID":"e334d773-74b6-41d4-b1bc-d0c0e4fb8f81","Type":"ContainerStarted","Data":"cbf39c08f5af08e73c67e900c5ef561f921fe3519996e99cf89b752275c2d352"} Jul 13 17:39:35.046965 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:35.046783 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" event={"ID":"e334d773-74b6-41d4-b1bc-d0c0e4fb8f81","Type":"ContainerStarted","Data":"e436950d3c5f68eb269847a0ff2306bf0f61f98cd6fa72d5abf2841795e5ac8f"} Jul 13 17:39:35.046965 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:35.046815 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:39:35.070806 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:35.070691 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" podStartSLOduration=1.070671291 podStartE2EDuration="1.070671291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:39:35.069077168 +0000 UTC m=+1313.225564045" watchObservedRunningTime="2026-07-13 17:39:35.070671291 +0000 UTC m=+1313.227158165" Jul 13 17:39:46.054609 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:39:46.054561 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-q2djd" Jul 13 17:42:42.609140 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:42:42.609112 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:42:42.613260 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:42:42.613237 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:45:57.169928 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.169891 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9"] Jul 13 17:45:57.174421 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.174386 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.176952 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.176931 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ec3f2398-openshift-default-dockercfg-ddtgh\"" Jul 13 17:45:57.190068 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.190039 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9"] Jul 13 17:45:57.286733 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.286690 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-podinfo\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.286888 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.286745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-data\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.286963 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.286936 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c86969-3afb-4ca6-8d34-a03bf3490363-istiod-ca-cert\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.287023 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.286979 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-credential-socket\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.287023 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.287009 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-certs\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.287118 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.287046 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-socket\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.287118 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.287069 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-envoy\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.287118 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.287114 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfqrx\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-kube-api-access-wfqrx\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.287229 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.287134 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-token\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.388639 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfqrx\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-kube-api-access-wfqrx\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.388779 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-token\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.388779 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388728 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-podinfo\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.388779 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388769 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-data\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.388933 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388806 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c86969-3afb-4ca6-8d34-a03bf3490363-istiod-ca-cert\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.388933 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388837 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-credential-socket\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.388933 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-certs\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.389102 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388965 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-socket\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.389102 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.388998 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-envoy\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.389271 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.389242 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-data\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.389395 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.389273 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-certs\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.389395 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.389362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-credential-socket\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.389503 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.389428 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-socket\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.389749 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.389729 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c86969-3afb-4ca6-8d34-a03bf3490363-istiod-ca-cert\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.391183 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.391164 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-envoy\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.391420 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.391399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-podinfo\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.399292 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.399271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-token\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.399901 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.399879 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfqrx\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-kube-api-access-wfqrx\") pod \"e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.486040 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.485984 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:57.635916 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.635886 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9"] Jul 13 17:45:57.636974 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:45:57.636948 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c86969_3afb_4ca6_8d34_a03bf3490363.slice/crio-0497fd2f39bb0e79574fb134180eaba27cf2da3341ed73cd0d807ba53a267558 WatchSource:0}: Error finding container 0497fd2f39bb0e79574fb134180eaba27cf2da3341ed73cd0d807ba53a267558: Status 404 returned error can't find the container with id 0497fd2f39bb0e79574fb134180eaba27cf2da3341ed73cd0d807ba53a267558 Jul 13 17:45:57.639340 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.639289 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:45:57.639410 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.639370 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:45:57.639410 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:57.639397 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:45:58.123405 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:58.123369 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" event={"ID":"c5c86969-3afb-4ca6-8d34-a03bf3490363","Type":"ContainerStarted","Data":"737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0"} Jul 13 17:45:58.123608 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:58.123422 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" event={"ID":"c5c86969-3afb-4ca6-8d34-a03bf3490363","Type":"ContainerStarted","Data":"0497fd2f39bb0e79574fb134180eaba27cf2da3341ed73cd0d807ba53a267558"} Jul 13 17:45:58.147911 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:58.147863 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" podStartSLOduration=1.14784832 podStartE2EDuration="1.14784832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:45:58.146009675 +0000 UTC m=+1696.302496553" watchObservedRunningTime="2026-07-13 17:45:58.14784832 +0000 UTC m=+1696.304335193" Jul 13 17:45:58.486389 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:58.486356 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:58.491480 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:58.491456 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:59.130633 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:59.130589 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:45:59.131688 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:45:59.131666 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:46:10.198642 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:10.198605 2579 generic.go:356] "Generic (PLEG): container finished" podID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerID="737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0" exitCode=0 Jul 13 17:46:10.198995 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:10.198668 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" event={"ID":"c5c86969-3afb-4ca6-8d34-a03bf3490363","Type":"ContainerDied","Data":"737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0"} Jul 13 17:46:10.199103 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:10.199083 2579 scope.go:117] "RemoveContainer" containerID="737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0" Jul 13 17:46:10.200081 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:10.200054 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:46:10.200152 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:10.200121 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:46:10.200195 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:10.200153 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:46:11.206933 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:11.206899 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" event={"ID":"c5c86969-3afb-4ca6-8d34-a03bf3490363","Type":"ContainerStarted","Data":"9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c"} Jul 13 17:46:11.486664 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:11.486569 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:46:11.487876 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:11.487849 2579 patch_prober.go:28] interesting pod/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 13 17:46:11.488007 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:11.487899 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 13 17:46:12.487428 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:12.487392 2579 patch_prober.go:28] interesting pod/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 13 17:46:12.487830 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:12.487462 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 13 17:46:13.486577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:13.486537 2579 patch_prober.go:28] interesting pod/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 13 17:46:13.486788 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:13.486593 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 13 17:46:14.131426 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:14.131391 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:46:14.486869 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:14.486792 2579 patch_prober.go:28] interesting pod/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 13 17:46:14.487013 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:14.486871 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 13 17:46:15.490304 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:15.490277 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:46:15.491341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:46:15.491307 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:47:28.849574 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:28.849535 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9"] Jul 13 17:47:28.850069 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:28.849795 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" containerID="cri-o://9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c" gracePeriod=30 Jul 13 17:47:29.400478 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.400436 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd"] Jul 13 17:47:29.404736 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.404709 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.407060 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.407036 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-35a3687f-gw-openshift-default-dockercfg-zdg7p\"" Jul 13 17:47:29.418099 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.418072 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd"] Jul 13 17:47:29.557928 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.557891 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-data\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.558107 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.557939 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-certs\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.558107 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.557974 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8lrk\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-kube-api-access-z8lrk\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.558107 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.558026 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-token\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.558107 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.558074 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-podinfo\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.558107 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.558098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-credential-socket\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.558277 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.558153 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-socket\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.558277 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.558191 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-envoy\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.558277 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.558237 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daac54f4-6f1c-465f-9d45-1366fa63a362-istiod-ca-cert\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.658977 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.658886 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daac54f4-6f1c-465f-9d45-1366fa63a362-istiod-ca-cert\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.658977 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.658936 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-data\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.658977 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.658969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-certs\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.658993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8lrk\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-kube-api-access-z8lrk\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659015 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-token\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659053 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-podinfo\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659082 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-credential-socket\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-socket\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-envoy\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659586 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659437 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-data\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659648 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659615 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-credential-socket\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659706 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659619 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-certs\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659706 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659669 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daac54f4-6f1c-465f-9d45-1366fa63a362-istiod-ca-cert\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.659706 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.659675 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-socket\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.661352 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.661313 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-envoy\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.661577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.661557 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-podinfo\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.668271 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.668249 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-token\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.668756 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.668740 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8lrk\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-kube-api-access-z8lrk\") pod \"e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.716594 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.716575 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:29.854859 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.854829 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd"] Jul 13 17:47:29.856381 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:47:29.856331 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaac54f4_6f1c_465f_9d45_1366fa63a362.slice/crio-e282ce0ed0b37abfb410a3ea389741229748006f59a3e14b4b8c476806bbad6f WatchSource:0}: Error finding container e282ce0ed0b37abfb410a3ea389741229748006f59a3e14b4b8c476806bbad6f: Status 404 returned error can't find the container with id e282ce0ed0b37abfb410a3ea389741229748006f59a3e14b4b8c476806bbad6f Jul 13 17:47:29.858808 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.858774 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:47:29.858905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.858840 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:47:29.858905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:29.858866 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:47:30.492913 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:30.492876 2579 patch_prober.go:28] interesting pod/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 17:47:30.493074 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:30.492929 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 17:47:30.676044 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:30.676012 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" event={"ID":"daac54f4-6f1c-465f-9d45-1366fa63a362","Type":"ContainerStarted","Data":"a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a"} Jul 13 17:47:30.676194 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:30.676068 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" event={"ID":"daac54f4-6f1c-465f-9d45-1366fa63a362","Type":"ContainerStarted","Data":"e282ce0ed0b37abfb410a3ea389741229748006f59a3e14b4b8c476806bbad6f"} Jul 13 17:47:30.697109 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:30.697054 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" podStartSLOduration=1.69703765 podStartE2EDuration="1.69703765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:47:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:47:30.696371714 +0000 UTC m=+1788.852858580" watchObservedRunningTime="2026-07-13 17:47:30.69703765 +0000 UTC m=+1788.853524527" Jul 13 17:47:30.717407 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:30.717384 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:30.722559 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:30.722538 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:31.682113 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:31.682070 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:31.682943 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:31.682924 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:34.112055 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.112006 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:47:34.196470 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196445 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-credential-socket\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196601 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196489 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-podinfo\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196601 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196538 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-socket\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196601 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196566 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c86969-3afb-4ca6-8d34-a03bf3490363-istiod-ca-cert\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196729 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196598 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-envoy\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196729 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196623 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-data\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196729 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196666 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfqrx\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-kube-api-access-wfqrx\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196868 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196729 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-certs\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196868 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196780 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-token\") pod \"c5c86969-3afb-4ca6-8d34-a03bf3490363\" (UID: \"c5c86969-3afb-4ca6-8d34-a03bf3490363\") " Jul 13 17:47:34.196868 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196726 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:47:34.196868 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.196796 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:47:34.197127 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.197102 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.197240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.197132 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.197312 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.197289 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:47:34.197398 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.197349 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5c86969-3afb-4ca6-8d34-a03bf3490363-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:47:34.197669 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.197648 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-data" (OuterVolumeSpecName: "istio-data") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:47:34.199096 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.199070 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-token" (OuterVolumeSpecName: "istio-token") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:47:34.199243 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.199220 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:47:34.199364 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.199336 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:47:34.199465 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.199412 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-kube-api-access-wfqrx" (OuterVolumeSpecName: "kube-api-access-wfqrx") pod "c5c86969-3afb-4ca6-8d34-a03bf3490363" (UID: "c5c86969-3afb-4ca6-8d34-a03bf3490363"). InnerVolumeSpecName "kube-api-access-wfqrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:47:34.298071 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.298040 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.298071 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.298067 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.298071 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.298076 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.298246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.298084 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c86969-3afb-4ca6-8d34-a03bf3490363-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.298246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.298092 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.298246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.298100 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c86969-3afb-4ca6-8d34-a03bf3490363-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.298246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.298108 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfqrx\" (UniqueName: \"kubernetes.io/projected/c5c86969-3afb-4ca6-8d34-a03bf3490363-kube-api-access-wfqrx\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:47:34.700152 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.700120 2579 generic.go:356] "Generic (PLEG): container finished" podID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerID="9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c" exitCode=0 Jul 13 17:47:34.700271 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.700168 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" Jul 13 17:47:34.700271 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.700199 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" event={"ID":"c5c86969-3afb-4ca6-8d34-a03bf3490363","Type":"ContainerDied","Data":"9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c"} Jul 13 17:47:34.700271 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.700259 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9" event={"ID":"c5c86969-3afb-4ca6-8d34-a03bf3490363","Type":"ContainerDied","Data":"0497fd2f39bb0e79574fb134180eaba27cf2da3341ed73cd0d807ba53a267558"} Jul 13 17:47:34.700422 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.700283 2579 scope.go:117] "RemoveContainer" containerID="9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c" Jul 13 17:47:34.709941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.709925 2579 scope.go:117] "RemoveContainer" containerID="737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0" Jul 13 17:47:34.718687 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.718670 2579 scope.go:117] "RemoveContainer" containerID="9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c" Jul 13 17:47:34.718852 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.718835 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9"] Jul 13 17:47:34.718939 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:47:34.718920 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c\": container with ID starting with 9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c not found: ID does not exist" containerID="9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c" Jul 13 17:47:34.718983 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.718949 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c"} err="failed to get container status \"9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c\": rpc error: code = NotFound desc = could not find container \"9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c\": container with ID starting with 9a1249381dcae5617ba94dc4c3f6a2ab3d09a805e0894aebff0e471c991af49c not found: ID does not exist" Jul 13 17:47:34.718983 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.718966 2579 scope.go:117] "RemoveContainer" containerID="737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0" Jul 13 17:47:34.719240 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:47:34.719223 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0\": container with ID starting with 737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0 not found: ID does not exist" containerID="737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0" Jul 13 17:47:34.719301 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.719247 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0"} err="failed to get container status \"737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0\": rpc error: code = NotFound desc = could not find container \"737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0\": container with ID starting with 737f6257fd7724c5e897ff659ff6e6fd05a8954bb372fad280793f54c3108ef0 not found: ID does not exist" Jul 13 17:47:34.722505 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:34.722485 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ec3f2398-openshift-default-64bdbd86f7-xnjl9"] Jul 13 17:47:36.661580 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:36.661544 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" path="/var/lib/kubelet/pods/c5c86969-3afb-4ca6-8d34-a03bf3490363/volumes" Jul 13 17:47:42.669475 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:42.669447 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:47:42.673266 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:42.673237 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:47:46.923222 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:46.921843 2579 patch_prober.go:28] interesting pod/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:35788->10.133.0.53:15021: read: connection reset by peer" start-of-body= Jul 13 17:47:46.923222 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:46.921918 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:35788->10.133.0.53:15021: read: connection reset by peer" Jul 13 17:47:47.786118 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:47.786081 2579 generic.go:356] "Generic (PLEG): container finished" podID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerID="a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a" exitCode=0 Jul 13 17:47:47.786118 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:47.786119 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" event={"ID":"daac54f4-6f1c-465f-9d45-1366fa63a362","Type":"ContainerDied","Data":"a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a"} Jul 13 17:47:47.786486 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:47.786471 2579 scope.go:117] "RemoveContainer" containerID="a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a" Jul 13 17:47:47.787431 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:47.787399 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:47:47.787509 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:47.787491 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:47:47.787550 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:47.787532 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:47:48.793655 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:48.793620 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" event={"ID":"daac54f4-6f1c-465f-9d45-1366fa63a362","Type":"ContainerStarted","Data":"17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f"} Jul 13 17:47:49.717028 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:49.716993 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:49.717198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:49.717114 2579 patch_prober.go:28] interesting pod/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 13 17:47:49.717198 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:49.717161 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 13 17:47:50.717782 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:50.717744 2579 patch_prober.go:28] interesting pod/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 13 17:47:50.718167 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:50.717800 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 13 17:47:51.717119 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:51.716992 2579 patch_prober.go:28] interesting pod/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 13 17:47:51.717119 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:51.717056 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 13 17:47:52.717050 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:52.717010 2579 patch_prober.go:28] interesting pod/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 13 17:47:52.717542 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:52.717070 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 13 17:47:53.720993 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:53.720964 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:53.721381 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:53.721353 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:47:53.722075 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:47:53.722058 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:48:35.254467 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.254433 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7"] Jul 13 17:48:35.255058 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.255040 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" Jul 13 17:48:35.255139 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.255064 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" Jul 13 17:48:35.255139 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.255081 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" Jul 13 17:48:35.255139 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.255090 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" Jul 13 17:48:35.255297 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.255189 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" Jul 13 17:48:35.258419 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.258396 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" Jul 13 17:48:35.261332 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.261302 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-prkcz\"" Jul 13 17:48:35.266162 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.266115 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7"] Jul 13 17:48:35.347895 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.347861 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd"] Jul 13 17:48:35.348085 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.348053 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" containerID="cri-o://17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f" gracePeriod=30 Jul 13 17:48:35.348406 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.348371 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dc835f-82b3-4131-a5a1-93fc14a17d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7\" (UID: \"d0dc835f-82b3-4131-a5a1-93fc14a17d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" Jul 13 17:48:35.449816 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.449781 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dc835f-82b3-4131-a5a1-93fc14a17d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7\" (UID: \"d0dc835f-82b3-4131-a5a1-93fc14a17d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" Jul 13 17:48:35.450533 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.450510 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dc835f-82b3-4131-a5a1-93fc14a17d52-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7\" (UID: \"d0dc835f-82b3-4131-a5a1-93fc14a17d52\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" Jul 13 17:48:35.569658 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.569566 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" Jul 13 17:48:35.723578 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.723546 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7"] Jul 13 17:48:35.726087 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:48:35.726053 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0dc835f_82b3_4131_a5a1_93fc14a17d52.slice/crio-fb8340a821a91b3853dd5300baf2e9864b5b98b570fd756a21b66ed9da873867 WatchSource:0}: Error finding container fb8340a821a91b3853dd5300baf2e9864b5b98b570fd756a21b66ed9da873867: Status 404 returned error can't find the container with id fb8340a821a91b3853dd5300baf2e9864b5b98b570fd756a21b66ed9da873867 Jul 13 17:48:35.728411 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:35.728395 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 17:48:36.066854 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:36.066821 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" event={"ID":"d0dc835f-82b3-4131-a5a1-93fc14a17d52","Type":"ContainerStarted","Data":"fb8340a821a91b3853dd5300baf2e9864b5b98b570fd756a21b66ed9da873867"} Jul 13 17:48:38.723406 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:38.723376 2579 patch_prober.go:28] interesting pod/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 17:48:38.723778 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:38.723432 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 17:48:39.086892 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:39.086808 2579 generic.go:356] "Generic (PLEG): container finished" podID="d0dc835f-82b3-4131-a5a1-93fc14a17d52" containerID="a366e71ffe564882853dfd8a5f7938b0bb83b5876ca5763cb1c5e179cc02f409" exitCode=0 Jul 13 17:48:39.087035 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:39.086893 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" event={"ID":"d0dc835f-82b3-4131-a5a1-93fc14a17d52","Type":"ContainerDied","Data":"a366e71ffe564882853dfd8a5f7938b0bb83b5876ca5763cb1c5e179cc02f409"} Jul 13 17:48:40.230145 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.230114 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" Jul 13 17:48:40.303125 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.303097 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dc835f-82b3-4131-a5a1-93fc14a17d52-maas-api-service-ca\") pod \"d0dc835f-82b3-4131-a5a1-93fc14a17d52\" (UID: \"d0dc835f-82b3-4131-a5a1-93fc14a17d52\") " Jul 13 17:48:40.303516 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.303490 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0dc835f-82b3-4131-a5a1-93fc14a17d52-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d0dc835f-82b3-4131-a5a1-93fc14a17d52" (UID: "d0dc835f-82b3-4131-a5a1-93fc14a17d52"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:48:40.404411 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.404390 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dc835f-82b3-4131-a5a1-93fc14a17d52-maas-api-service-ca\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.597406 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.597378 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:48:40.706339 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706293 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-podinfo\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706445 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706355 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-certs\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706445 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706390 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8lrk\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-kube-api-access-z8lrk\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706445 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706407 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daac54f4-6f1c-465f-9d45-1366fa63a362-istiod-ca-cert\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706590 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706548 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-socket\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706655 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706603 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-data\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706655 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706638 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-envoy\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706754 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706676 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-credential-socket\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706754 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706706 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:40.706754 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706726 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-token\") pod \"daac54f4-6f1c-465f-9d45-1366fa63a362\" (UID: \"daac54f4-6f1c-465f-9d45-1366fa63a362\") " Jul 13 17:48:40.706942 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706903 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daac54f4-6f1c-465f-9d45-1366fa63a362-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:48:40.706942 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.706924 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:40.707095 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.707079 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.707164 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.707101 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daac54f4-6f1c-465f-9d45-1366fa63a362-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.707164 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.707115 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.707432 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.707405 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:40.707677 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.707641 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-data" (OuterVolumeSpecName: "istio-data") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:40.708701 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.708677 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:48:40.708796 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.708720 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-kube-api-access-z8lrk" (OuterVolumeSpecName: "kube-api-access-z8lrk") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "kube-api-access-z8lrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:48:40.708796 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.708744 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:48:40.708882 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.708802 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-token" (OuterVolumeSpecName: "istio-token") pod "daac54f4-6f1c-465f-9d45-1366fa63a362" (UID: "daac54f4-6f1c-465f-9d45-1366fa63a362"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:48:40.808129 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.808105 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.808129 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.808127 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8lrk\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-kube-api-access-z8lrk\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.808294 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.808137 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.808294 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.808147 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.808294 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.808155 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:40.808294 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:40.808162 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daac54f4-6f1c-465f-9d45-1366fa63a362-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:48:41.102610 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.102521 2579 generic.go:356] "Generic (PLEG): container finished" podID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerID="17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f" exitCode=0 Jul 13 17:48:41.102610 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.102574 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" Jul 13 17:48:41.102832 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.102606 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" event={"ID":"daac54f4-6f1c-465f-9d45-1366fa63a362","Type":"ContainerDied","Data":"17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f"} Jul 13 17:48:41.102832 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.102676 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd" event={"ID":"daac54f4-6f1c-465f-9d45-1366fa63a362","Type":"ContainerDied","Data":"e282ce0ed0b37abfb410a3ea389741229748006f59a3e14b4b8c476806bbad6f"} Jul 13 17:48:41.102832 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.102715 2579 scope.go:117] "RemoveContainer" containerID="17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f" Jul 13 17:48:41.104828 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.104784 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" event={"ID":"d0dc835f-82b3-4131-a5a1-93fc14a17d52","Type":"ContainerDied","Data":"fb8340a821a91b3853dd5300baf2e9864b5b98b570fd756a21b66ed9da873867"} Jul 13 17:48:41.104961 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.104830 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb8340a821a91b3853dd5300baf2e9864b5b98b570fd756a21b66ed9da873867" Jul 13 17:48:41.104961 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.104795 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7" Jul 13 17:48:41.116494 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.116472 2579 scope.go:117] "RemoveContainer" containerID="a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a" Jul 13 17:48:41.125961 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.125933 2579 scope.go:117] "RemoveContainer" containerID="17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f" Jul 13 17:48:41.126271 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:48:41.126248 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f\": container with ID starting with 17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f not found: ID does not exist" containerID="17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f" Jul 13 17:48:41.126363 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.126281 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f"} err="failed to get container status \"17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f\": rpc error: code = NotFound desc = could not find container \"17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f\": container with ID starting with 17881e9e90a0b156fe7a6ed06f17f9c6fa1fd27c64f2ac9f90c678613618ab0f not found: ID does not exist" Jul 13 17:48:41.126363 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.126303 2579 scope.go:117] "RemoveContainer" containerID="a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a" Jul 13 17:48:41.126688 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:48:41.126668 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a\": container with ID starting with a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a not found: ID does not exist" containerID="a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a" Jul 13 17:48:41.126753 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.126694 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a"} err="failed to get container status \"a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a\": rpc error: code = NotFound desc = could not find container \"a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a\": container with ID starting with a39b0587eadffe0c78b872f24e9c3df90e3341f1812c429569cbe236a7abc36a not found: ID does not exist" Jul 13 17:48:41.134905 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.134881 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd"] Jul 13 17:48:41.137312 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:41.137293 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-35a3687f-gw-openshift-default-6df7d5f579-9dzwd"] Jul 13 17:48:42.662667 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:42.662636 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" path="/var/lib/kubelet/pods/daac54f4-6f1c-465f-9d45-1366fa63a362/volumes" Jul 13 17:48:48.353818 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.353793 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p"] Jul 13 17:48:48.354219 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354200 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0dc835f-82b3-4131-a5a1-93fc14a17d52" containerName="revoke-keys" Jul 13 17:48:48.354219 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354216 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0dc835f-82b3-4131-a5a1-93fc14a17d52" containerName="revoke-keys" Jul 13 17:48:48.354439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354247 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" Jul 13 17:48:48.354439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354254 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" Jul 13 17:48:48.354439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354265 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" Jul 13 17:48:48.354439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354272 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" Jul 13 17:48:48.354439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354374 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0dc835f-82b3-4131-a5a1-93fc14a17d52" containerName="revoke-keys" Jul 13 17:48:48.354439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354388 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" Jul 13 17:48:48.354439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354400 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c86969-3afb-4ca6-8d34-a03bf3490363" containerName="istio-proxy" Jul 13 17:48:48.354439 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.354412 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="daac54f4-6f1c-465f-9d45-1366fa63a362" containerName="istio-proxy" Jul 13 17:48:48.357811 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.357791 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.360484 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.360464 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d0c7b7fa-openshift-default-dockercfg-222lq\"" Jul 13 17:48:48.373843 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.373817 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p"] Jul 13 17:48:48.467564 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467526 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istiod-ca-cert\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.467716 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467642 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phlmv\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-kube-api-access-phlmv\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.467785 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467724 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-data\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.467785 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467755 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-certs\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.467892 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467795 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-envoy\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.467892 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-socket\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.467892 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467870 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-podinfo\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.468045 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467898 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-credential-socket\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.468045 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.467927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-token\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.568604 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568564 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-socket\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.568604 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-podinfo\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.568829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568629 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-credential-socket\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.568829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568648 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-token\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.568829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568676 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istiod-ca-cert\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.568829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phlmv\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-kube-api-access-phlmv\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.568829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568760 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-data\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.568829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568793 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-certs\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.569130 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568833 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-envoy\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.569130 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.568984 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-socket\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.569130 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.569036 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-credential-socket\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.569289 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.569203 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-data\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.569342 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.569306 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-certs\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.569633 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.569612 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istiod-ca-cert\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.571043 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.571020 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-envoy\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.571185 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.571167 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-podinfo\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.588570 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.588543 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phlmv\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-kube-api-access-phlmv\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.590431 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.590411 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-token\") pod \"e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.667895 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.667873 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:48.812011 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.811982 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p"] Jul 13 17:48:48.813291 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:48:48.813264 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97cfa0bf_cdb2_4036_95d1_ddaec4b8fb93.slice/crio-b5b98ac67e14140234305e09529eb924ad4da3b46692fc9ea2552a1fe552367b WatchSource:0}: Error finding container b5b98ac67e14140234305e09529eb924ad4da3b46692fc9ea2552a1fe552367b: Status 404 returned error can't find the container with id b5b98ac67e14140234305e09529eb924ad4da3b46692fc9ea2552a1fe552367b Jul 13 17:48:48.815454 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.815421 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:48:48.815526 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.815502 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:48:48.815561 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:48.815532 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:48:49.149610 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:49.149576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" event={"ID":"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93","Type":"ContainerStarted","Data":"5dcdd1a39c7ca9fe2e17e74aade236b8662d84535be36765e5895733bfaf3f1e"} Jul 13 17:48:49.149759 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:49.149633 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" event={"ID":"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93","Type":"ContainerStarted","Data":"b5b98ac67e14140234305e09529eb924ad4da3b46692fc9ea2552a1fe552367b"} Jul 13 17:48:49.183664 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:49.183618 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" podStartSLOduration=1.183600819 podStartE2EDuration="1.183600819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:48:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:48:49.179401756 +0000 UTC m=+1867.335888631" watchObservedRunningTime="2026-07-13 17:48:49.183600819 +0000 UTC m=+1867.340087694" Jul 13 17:48:49.668363 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:49.668314 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:49.673049 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:49.673019 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:50.154755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:50.154718 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:48:50.155738 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:48:50.155720 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:49:00.211009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:00.210915 2579 generic.go:356] "Generic (PLEG): container finished" podID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerID="5dcdd1a39c7ca9fe2e17e74aade236b8662d84535be36765e5895733bfaf3f1e" exitCode=0 Jul 13 17:49:00.211009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:00.210981 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" event={"ID":"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93","Type":"ContainerDied","Data":"5dcdd1a39c7ca9fe2e17e74aade236b8662d84535be36765e5895733bfaf3f1e"} Jul 13 17:49:00.211593 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:00.211514 2579 scope.go:117] "RemoveContainer" containerID="5dcdd1a39c7ca9fe2e17e74aade236b8662d84535be36765e5895733bfaf3f1e" Jul 13 17:49:00.212569 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:00.212536 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:49:00.212642 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:00.212612 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:49:00.212679 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:00.212650 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:49:01.217974 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:01.217940 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" event={"ID":"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93","Type":"ContainerStarted","Data":"7bf2f20c68388d91a82fcf324060c6d1e59f438ba443a6124f031884aeb58663"} Jul 13 17:49:01.668375 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:01.668309 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:49:01.669627 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:01.669598 2579 patch_prober.go:28] interesting pod/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 13 17:49:01.669752 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:01.669650 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 13 17:49:02.668942 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:02.668907 2579 patch_prober.go:28] interesting pod/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 13 17:49:02.669347 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:02.668961 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 13 17:49:03.668482 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:03.668441 2579 patch_prober.go:28] interesting pod/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 13 17:49:03.668682 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:03.668507 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 13 17:49:04.668639 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:04.668603 2579 patch_prober.go:28] interesting pod/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 13 17:49:04.669052 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:04.668656 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 13 17:49:05.154654 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:05.154612 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:49:05.672108 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:05.672074 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:49:05.673175 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:49:05.673151 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:50:21.512344 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:21.512288 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p"] Jul 13 17:50:21.513755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:21.513679 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" containerID="cri-o://7bf2f20c68388d91a82fcf324060c6d1e59f438ba443a6124f031884aeb58663" gracePeriod=30 Jul 13 17:50:21.891274 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:21.891239 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h"] Jul 13 17:50:21.895598 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:21.895575 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:21.899658 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:21.899638 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7c24d8cc-openshift-default-dockercfg-clwsb\"" Jul 13 17:50:21.909183 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:21.909153 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h"] Jul 13 17:50:22.033858 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.033817 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-credential-socket\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.034027 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.033866 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-envoy\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.034027 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.033957 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-certs\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.034027 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.033983 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-data\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.034027 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.034007 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-istio-token\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.034188 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.034105 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374efeb8-2ccc-424e-9729-8b8362c78259-istiod-ca-cert\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.034188 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.034170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374efeb8-2ccc-424e-9729-8b8362c78259-istio-podinfo\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.034291 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.034245 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-socket\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.034385 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.034300 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzkrc\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-kube-api-access-tzkrc\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135145 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzkrc\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-kube-api-access-tzkrc\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135299 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135155 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-credential-socket\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135299 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135172 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-envoy\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135470 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-certs\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135470 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135363 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-data\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135470 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135392 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-istio-token\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135470 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135453 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374efeb8-2ccc-424e-9729-8b8362c78259-istiod-ca-cert\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135704 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374efeb8-2ccc-424e-9729-8b8362c78259-istio-podinfo\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135704 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135565 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-socket\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135704 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135660 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-credential-socket\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135860 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135794 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-data\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.135974 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135956 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-socket\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.136035 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.135992 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-certs\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.136360 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.136334 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374efeb8-2ccc-424e-9729-8b8362c78259-istiod-ca-cert\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.137528 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.137502 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-envoy\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.138131 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.138110 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374efeb8-2ccc-424e-9729-8b8362c78259-istio-podinfo\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.143454 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.143401 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-istio-token\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.143709 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.143693 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzkrc\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-kube-api-access-tzkrc\") pod \"e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.208202 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.208178 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:22.356531 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.356504 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h"] Jul 13 17:50:22.357594 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:50:22.357568 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374efeb8_2ccc_424e_9729_8b8362c78259.slice/crio-bd150003007a1c7c67f13184485f2a0daecbadaf81339a1ab501d1a860e94c5b WatchSource:0}: Error finding container bd150003007a1c7c67f13184485f2a0daecbadaf81339a1ab501d1a860e94c5b: Status 404 returned error can't find the container with id bd150003007a1c7c67f13184485f2a0daecbadaf81339a1ab501d1a860e94c5b Jul 13 17:50:22.360256 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.360219 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:50:22.360376 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.360304 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:50:22.360376 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.360369 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:50:22.677124 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.677047 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" event={"ID":"374efeb8-2ccc-424e-9729-8b8362c78259","Type":"ContainerStarted","Data":"74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af"} Jul 13 17:50:22.677124 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.677096 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" event={"ID":"374efeb8-2ccc-424e-9729-8b8362c78259","Type":"ContainerStarted","Data":"bd150003007a1c7c67f13184485f2a0daecbadaf81339a1ab501d1a860e94c5b"} Jul 13 17:50:22.699227 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:22.699181 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" podStartSLOduration=1.699165638 podStartE2EDuration="1.699165638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:50:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:50:22.6964391 +0000 UTC m=+1960.852925980" watchObservedRunningTime="2026-07-13 17:50:22.699165638 +0000 UTC m=+1960.855652513" Jul 13 17:50:23.208376 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:23.208344 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:23.213259 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:23.213234 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:23.682644 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:23.682612 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:23.683610 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:23.683592 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:26.703829 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.703791 2579 generic.go:356] "Generic (PLEG): container finished" podID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerID="7bf2f20c68388d91a82fcf324060c6d1e59f438ba443a6124f031884aeb58663" exitCode=0 Jul 13 17:50:26.704589 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.703860 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" event={"ID":"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93","Type":"ContainerDied","Data":"7bf2f20c68388d91a82fcf324060c6d1e59f438ba443a6124f031884aeb58663"} Jul 13 17:50:26.704589 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.703919 2579 scope.go:117] "RemoveContainer" containerID="5dcdd1a39c7ca9fe2e17e74aade236b8662d84535be36765e5895733bfaf3f1e" Jul 13 17:50:26.780031 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.780005 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:50:26.876746 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.876721 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-data\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.876888 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.876756 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-credential-socket\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.876888 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.876791 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-certs\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.876888 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.876857 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-envoy\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.876888 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.876885 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-podinfo\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.877105 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.876935 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phlmv\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-kube-api-access-phlmv\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.877105 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.876970 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-socket\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.877105 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.876998 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-token\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.877105 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877023 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istiod-ca-cert\") pod \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\" (UID: \"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93\") " Jul 13 17:50:26.877105 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877037 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:26.877387 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877104 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:26.877387 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877266 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:26.877498 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877466 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:26.877498 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877488 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:26.877606 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877506 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:26.877659 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877597 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:50:26.877729 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.877703 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-data" (OuterVolumeSpecName: "istio-data") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:26.879026 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.879000 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:50:26.879694 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.879664 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:50:26.879694 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.879670 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-kube-api-access-phlmv" (OuterVolumeSpecName: "kube-api-access-phlmv") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "kube-api-access-phlmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:50:26.879834 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.879696 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-token" (OuterVolumeSpecName: "istio-token") pod "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" (UID: "97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:50:26.978512 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.978482 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:26.978512 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.978510 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:26.978679 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.978523 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:26.978679 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.978537 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phlmv\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-kube-api-access-phlmv\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:26.978679 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.978552 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:26.978679 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:26.978565 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:50:27.711519 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:27.711490 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" Jul 13 17:50:27.711931 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:27.711537 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p" event={"ID":"97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93","Type":"ContainerDied","Data":"b5b98ac67e14140234305e09529eb924ad4da3b46692fc9ea2552a1fe552367b"} Jul 13 17:50:27.711931 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:27.711584 2579 scope.go:117] "RemoveContainer" containerID="7bf2f20c68388d91a82fcf324060c6d1e59f438ba443a6124f031884aeb58663" Jul 13 17:50:27.734996 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:27.734965 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p"] Jul 13 17:50:27.738882 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:27.738858 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d0c7b7fa-openshift-default-77b96fc8cf-fmg6p"] Jul 13 17:50:28.661211 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:28.661180 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" path="/var/lib/kubelet/pods/97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93/volumes" Jul 13 17:50:39.724523 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:39.724478 2579 patch_prober.go:28] interesting pod/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 17:50:39.724978 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:39.724553 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 17:50:40.791718 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:40.791681 2579 generic.go:356] "Generic (PLEG): container finished" podID="374efeb8-2ccc-424e-9729-8b8362c78259" containerID="74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af" exitCode=0 Jul 13 17:50:40.792244 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:40.791760 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" event={"ID":"374efeb8-2ccc-424e-9729-8b8362c78259","Type":"ContainerDied","Data":"74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af"} Jul 13 17:50:40.792244 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:40.792098 2579 scope.go:117] "RemoveContainer" containerID="74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af" Jul 13 17:50:40.793039 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:40.793000 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:50:40.793128 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:40.793083 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:50:40.793128 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:40.793117 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:50:41.801408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:41.801373 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" event={"ID":"374efeb8-2ccc-424e-9729-8b8362c78259","Type":"ContainerStarted","Data":"7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556"} Jul 13 17:50:42.209040 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:42.209006 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:42.209225 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:42.209196 2579 patch_prober.go:28] interesting pod/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 13 17:50:42.209273 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:42.209238 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 13 17:50:43.208988 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:43.208951 2579 patch_prober.go:28] interesting pod/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 13 17:50:43.209380 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:43.208999 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 13 17:50:44.208944 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:44.208904 2579 patch_prober.go:28] interesting pod/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 13 17:50:44.209120 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:44.208960 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 13 17:50:45.208884 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:45.208847 2579 patch_prober.go:28] interesting pod/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 13 17:50:45.209054 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:45.208900 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 13 17:50:46.212057 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:46.212025 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:46.212456 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:46.212262 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:50:46.212976 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:50:46.212957 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:51:28.501291 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.501248 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf"] Jul 13 17:51:28.501902 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.501881 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" Jul 13 17:51:28.501982 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.501905 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" Jul 13 17:51:28.501982 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.501920 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" Jul 13 17:51:28.501982 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.501928 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" Jul 13 17:51:28.502134 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.502037 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" Jul 13 17:51:28.502134 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.502056 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="97cfa0bf-cdb2-4036-95d1-ddaec4b8fb93" containerName="istio-proxy" Jul 13 17:51:28.505224 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.505198 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" Jul 13 17:51:28.505530 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.505509 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68be85d-b295-4600-b10f-7d5a64edb27f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf\" (UID: \"f68be85d-b295-4600-b10f-7d5a64edb27f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" Jul 13 17:51:28.507533 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.507516 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-prkcz\"" Jul 13 17:51:28.514722 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.514694 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf"] Jul 13 17:51:28.606128 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.606102 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68be85d-b295-4600-b10f-7d5a64edb27f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf\" (UID: \"f68be85d-b295-4600-b10f-7d5a64edb27f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" Jul 13 17:51:28.606705 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.606688 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68be85d-b295-4600-b10f-7d5a64edb27f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf\" (UID: \"f68be85d-b295-4600-b10f-7d5a64edb27f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" Jul 13 17:51:28.820990 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.820928 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" Jul 13 17:51:28.962117 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:28.962087 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf"] Jul 13 17:51:28.962961 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:51:28.962937 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf68be85d_b295_4600_b10f_7d5a64edb27f.slice/crio-a68d3a15c4177a9a8eb55d6201761a8dca9d001e8806a283744835b2f8d35875 WatchSource:0}: Error finding container a68d3a15c4177a9a8eb55d6201761a8dca9d001e8806a283744835b2f8d35875: Status 404 returned error can't find the container with id a68d3a15c4177a9a8eb55d6201761a8dca9d001e8806a283744835b2f8d35875 Jul 13 17:51:29.053069 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:29.053036 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" event={"ID":"f68be85d-b295-4600-b10f-7d5a64edb27f","Type":"ContainerStarted","Data":"a68d3a15c4177a9a8eb55d6201761a8dca9d001e8806a283744835b2f8d35875"} Jul 13 17:51:30.061757 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:30.061680 2579 generic.go:356] "Generic (PLEG): container finished" podID="f68be85d-b295-4600-b10f-7d5a64edb27f" containerID="fbf5ce05bcae6203195254dabddcc5050f8b2afa96a2897bcee7ecba93155585" exitCode=0 Jul 13 17:51:30.062084 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:30.061762 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" event={"ID":"f68be85d-b295-4600-b10f-7d5a64edb27f","Type":"ContainerDied","Data":"fbf5ce05bcae6203195254dabddcc5050f8b2afa96a2897bcee7ecba93155585"} Jul 13 17:51:31.200100 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:31.200076 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" Jul 13 17:51:31.227245 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:31.227219 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68be85d-b295-4600-b10f-7d5a64edb27f-maas-api-service-ca\") pod \"f68be85d-b295-4600-b10f-7d5a64edb27f\" (UID: \"f68be85d-b295-4600-b10f-7d5a64edb27f\") " Jul 13 17:51:31.227671 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:31.227642 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f68be85d-b295-4600-b10f-7d5a64edb27f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f68be85d-b295-4600-b10f-7d5a64edb27f" (UID: "f68be85d-b295-4600-b10f-7d5a64edb27f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:51:31.328752 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:31.328685 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f68be85d-b295-4600-b10f-7d5a64edb27f-maas-api-service-ca\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:32.074514 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:32.074485 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" Jul 13 17:51:32.074514 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:32.074505 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf" event={"ID":"f68be85d-b295-4600-b10f-7d5a64edb27f","Type":"ContainerDied","Data":"a68d3a15c4177a9a8eb55d6201761a8dca9d001e8806a283744835b2f8d35875"} Jul 13 17:51:32.074714 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:32.074549 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a68d3a15c4177a9a8eb55d6201761a8dca9d001e8806a283744835b2f8d35875" Jul 13 17:51:53.766956 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:53.766914 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h"] Jul 13 17:51:53.767507 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:53.767166 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" containerID="cri-o://7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556" gracePeriod=30 Jul 13 17:51:59.026128 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.026105 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:51:59.145977 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.145950 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-envoy\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146101 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.145983 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374efeb8-2ccc-424e-9729-8b8362c78259-istio-podinfo\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146101 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146010 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-certs\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146177 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146097 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-socket\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146177 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146142 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-data\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146280 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146169 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-istio-token\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146280 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146214 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-credential-socket\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146280 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146271 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374efeb8-2ccc-424e-9729-8b8362c78259-istiod-ca-cert\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146472 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146303 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzkrc\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-kube-api-access-tzkrc\") pod \"374efeb8-2ccc-424e-9729-8b8362c78259\" (UID: \"374efeb8-2ccc-424e-9729-8b8362c78259\") " Jul 13 17:51:59.146472 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146362 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:59.146472 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146392 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:59.146637 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146581 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:59.146717 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146692 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374efeb8-2ccc-424e-9729-8b8362c78259-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:51:59.146826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146699 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.146826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146743 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.146826 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.146762 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.147048 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.147026 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-data" (OuterVolumeSpecName: "istio-data") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:59.148213 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.148184 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:51:59.148349 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.148305 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-istio-token" (OuterVolumeSpecName: "istio-token") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:59.148493 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.148474 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/374efeb8-2ccc-424e-9729-8b8362c78259-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:51:59.148641 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.148627 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-kube-api-access-tzkrc" (OuterVolumeSpecName: "kube-api-access-tzkrc") pod "374efeb8-2ccc-424e-9729-8b8362c78259" (UID: "374efeb8-2ccc-424e-9729-8b8362c78259"). InnerVolumeSpecName "kube-api-access-tzkrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:51:59.224816 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.224790 2579 generic.go:356] "Generic (PLEG): container finished" podID="374efeb8-2ccc-424e-9729-8b8362c78259" containerID="7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556" exitCode=0 Jul 13 17:51:59.224934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.224844 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" event={"ID":"374efeb8-2ccc-424e-9729-8b8362c78259","Type":"ContainerDied","Data":"7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556"} Jul 13 17:51:59.224934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.224869 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" Jul 13 17:51:59.224934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.224894 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h" event={"ID":"374efeb8-2ccc-424e-9729-8b8362c78259","Type":"ContainerDied","Data":"bd150003007a1c7c67f13184485f2a0daecbadaf81339a1ab501d1a860e94c5b"} Jul 13 17:51:59.224934 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.224916 2579 scope.go:117] "RemoveContainer" containerID="7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556" Jul 13 17:51:59.233916 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.233898 2579 scope.go:117] "RemoveContainer" containerID="74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af" Jul 13 17:51:59.242784 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.242765 2579 scope.go:117] "RemoveContainer" containerID="7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556" Jul 13 17:51:59.243047 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:51:59.243028 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556\": container with ID starting with 7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556 not found: ID does not exist" containerID="7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556" Jul 13 17:51:59.243096 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.243056 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556"} err="failed to get container status \"7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556\": rpc error: code = NotFound desc = could not find container \"7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556\": container with ID starting with 7d7d92681413d0499c21fd9250ef4c48d731e1448f7a85c7e2aaec2503936556 not found: ID does not exist" Jul 13 17:51:59.243096 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.243074 2579 scope.go:117] "RemoveContainer" containerID="74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af" Jul 13 17:51:59.243311 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:51:59.243295 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af\": container with ID starting with 74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af not found: ID does not exist" containerID="74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af" Jul 13 17:51:59.243396 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.243328 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af"} err="failed to get container status \"74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af\": rpc error: code = NotFound desc = could not find container \"74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af\": container with ID starting with 74226b31c3eb241cba9f10e1fbc7e2407fabd6c34935cd53b9664dd9a8aa08af not found: ID does not exist" Jul 13 17:51:59.253359 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.248455 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzkrc\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-kube-api-access-tzkrc\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.253359 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.248501 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.253359 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.248518 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/374efeb8-2ccc-424e-9729-8b8362c78259-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.253359 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.248532 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/374efeb8-2ccc-424e-9729-8b8362c78259-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.253359 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.248546 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/374efeb8-2ccc-424e-9729-8b8362c78259-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.253359 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.248563 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/374efeb8-2ccc-424e-9729-8b8362c78259-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:51:59.256642 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.256619 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h"] Jul 13 17:51:59.259713 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:51:59.259691 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7c24d8cc-openshift-default-77595c6c9-vn84h"] Jul 13 17:52:00.662355 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:52:00.662304 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" path="/var/lib/kubelet/pods/374efeb8-2ccc-424e-9729-8b8362c78259/volumes" Jul 13 17:52:42.719012 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:52:42.718985 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:52:42.722714 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:52:42.722691 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:53:32.528838 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.528789 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8"] Jul 13 17:53:32.531401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.529779 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" Jul 13 17:53:32.531401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.529814 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" Jul 13 17:53:32.531401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.529836 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f68be85d-b295-4600-b10f-7d5a64edb27f" containerName="revoke-keys" Jul 13 17:53:32.531401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.529846 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f68be85d-b295-4600-b10f-7d5a64edb27f" containerName="revoke-keys" Jul 13 17:53:32.531401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.529865 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" Jul 13 17:53:32.531401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.529875 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" Jul 13 17:53:32.531401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.530073 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" Jul 13 17:53:32.531401 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.530088 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f68be85d-b295-4600-b10f-7d5a64edb27f" containerName="revoke-keys" Jul 13 17:53:32.534438 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.534403 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.538750 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.538722 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-a287d56e-openshift-default-dockercfg-zmmw4\"" Jul 13 17:53:32.551993 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.551966 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8"] Jul 13 17:53:32.609881 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.609838 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a80792f0-2878-4c2f-94d7-14edc640333e-istio-podinfo\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.610073 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.609894 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-credential-socket\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.610073 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.609919 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-data\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.610073 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.609976 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-istio-token\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.610219 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.610066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-envoy\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.610219 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.610116 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a80792f0-2878-4c2f-94d7-14edc640333e-istiod-ca-cert\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.610219 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.610178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-socket\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.610370 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.610230 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-certs\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.610370 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.610262 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw6nl\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-kube-api-access-cw6nl\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.710884 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.710849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-envoy\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.710884 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.710890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a80792f0-2878-4c2f-94d7-14edc640333e-istiod-ca-cert\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.710927 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-socket\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.710945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-certs\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.710966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw6nl\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-kube-api-access-cw6nl\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711006 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a80792f0-2878-4c2f-94d7-14edc640333e-istio-podinfo\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711025 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-credential-socket\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711046 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-data\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711099 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-istio-token\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711395 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-certs\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711696 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-data\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711831 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711719 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-socket\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711831 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711754 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a80792f0-2878-4c2f-94d7-14edc640333e-istiod-ca-cert\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.711930 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.711758 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-credential-socket\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.713354 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.713308 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-envoy\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.713571 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.713553 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a80792f0-2878-4c2f-94d7-14edc640333e-istio-podinfo\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.722813 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.722791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-istio-token\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.722947 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.722929 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw6nl\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-kube-api-access-cw6nl\") pod \"e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.845492 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.845408 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:32.995221 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.995193 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8"] Jul 13 17:53:32.996469 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:53:32.996441 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda80792f0_2878_4c2f_94d7_14edc640333e.slice/crio-a21847594a40108e73fd8f0e44360368be576c045776d784d34d7b7514917f70 WatchSource:0}: Error finding container a21847594a40108e73fd8f0e44360368be576c045776d784d34d7b7514917f70: Status 404 returned error can't find the container with id a21847594a40108e73fd8f0e44360368be576c045776d784d34d7b7514917f70 Jul 13 17:53:32.998577 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.998545 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:53:32.998677 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.998641 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:53:32.998742 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:32.998689 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:53:33.762765 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:33.762730 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" event={"ID":"a80792f0-2878-4c2f-94d7-14edc640333e","Type":"ContainerStarted","Data":"3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3"} Jul 13 17:53:33.763174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:33.762794 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" event={"ID":"a80792f0-2878-4c2f-94d7-14edc640333e","Type":"ContainerStarted","Data":"a21847594a40108e73fd8f0e44360368be576c045776d784d34d7b7514917f70"} Jul 13 17:53:33.789970 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:33.789918 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" podStartSLOduration=1.789903654 podStartE2EDuration="1.789903654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:53:33.787205919 +0000 UTC m=+2151.943692784" watchObservedRunningTime="2026-07-13 17:53:33.789903654 +0000 UTC m=+2151.946390532" Jul 13 17:53:33.846087 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:33.846047 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:34.851174 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:34.851139 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:35.773489 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:35.773453 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:35.774299 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:35.774281 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:48.532370 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.532334 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl"] Jul 13 17:53:48.533039 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.533019 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="374efeb8-2ccc-424e-9729-8b8362c78259" containerName="istio-proxy" Jul 13 17:53:48.536540 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.536524 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" Jul 13 17:53:48.545740 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.545712 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-prkcz\"" Jul 13 17:53:48.590827 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.590793 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl"] Jul 13 17:53:48.661603 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.661561 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a3fdb4-5c84-4e86-a3ed-bdcea7759da2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a287d56e-8gptl\" (UID: \"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" Jul 13 17:53:48.762290 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.762244 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a3fdb4-5c84-4e86-a3ed-bdcea7759da2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a287d56e-8gptl\" (UID: \"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" Jul 13 17:53:48.762939 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.762915 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a3fdb4-5c84-4e86-a3ed-bdcea7759da2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-a287d56e-8gptl\" (UID: \"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" Jul 13 17:53:48.850621 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:48.850539 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" Jul 13 17:53:49.003592 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:49.003558 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl"] Jul 13 17:53:49.007090 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:53:49.007053 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a3fdb4_5c84_4e86_a3ed_bdcea7759da2.slice/crio-38007340a90b061593f416eb88fdf48837cbd88d0a62357e68fd5c74bc15b098 WatchSource:0}: Error finding container 38007340a90b061593f416eb88fdf48837cbd88d0a62357e68fd5c74bc15b098: Status 404 returned error can't find the container with id 38007340a90b061593f416eb88fdf48837cbd88d0a62357e68fd5c74bc15b098 Jul 13 17:53:49.861309 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:49.861269 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" event={"ID":"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2","Type":"ContainerStarted","Data":"dd3e748a344a10487fb6f30f7ccc5505c28c1ed0e8d99783bbf2b23a7444a6f2"} Jul 13 17:53:49.861732 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:49.861351 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" event={"ID":"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2","Type":"ContainerStarted","Data":"38007340a90b061593f416eb88fdf48837cbd88d0a62357e68fd5c74bc15b098"} Jul 13 17:53:49.878755 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:49.878709 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" podStartSLOduration=1.878694963 podStartE2EDuration="1.878694963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:53:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:53:49.877159655 +0000 UTC m=+2168.033646522" watchObservedRunningTime="2026-07-13 17:53:49.878694963 +0000 UTC m=+2168.035181837" Jul 13 17:53:50.774859 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.774828 2579 patch_prober.go:28] interesting pod/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 13 17:53:50.775003 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.774883 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 13 17:53:50.869375 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.869285 2579 generic.go:356] "Generic (PLEG): container finished" podID="99a3fdb4-5c84-4e86-a3ed-bdcea7759da2" containerID="dd3e748a344a10487fb6f30f7ccc5505c28c1ed0e8d99783bbf2b23a7444a6f2" exitCode=0 Jul 13 17:53:50.869760 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.869364 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" event={"ID":"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2","Type":"ContainerDied","Data":"dd3e748a344a10487fb6f30f7ccc5505c28c1ed0e8d99783bbf2b23a7444a6f2"} Jul 13 17:53:50.871221 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.871198 2579 generic.go:356] "Generic (PLEG): container finished" podID="a80792f0-2878-4c2f-94d7-14edc640333e" containerID="3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3" exitCode=0 Jul 13 17:53:50.871345 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.871238 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" event={"ID":"a80792f0-2878-4c2f-94d7-14edc640333e","Type":"ContainerDied","Data":"3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3"} Jul 13 17:53:50.871746 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.871727 2579 scope.go:117] "RemoveContainer" containerID="3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3" Jul 13 17:53:50.872923 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.872885 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:53:50.872999 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.872980 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:53:50.873057 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:50.873025 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:53:51.879381 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:51.879339 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" event={"ID":"a80792f0-2878-4c2f-94d7-14edc640333e","Type":"ContainerStarted","Data":"1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a"} Jul 13 17:53:52.060024 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.059996 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" Jul 13 17:53:52.194842 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.194750 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a3fdb4-5c84-4e86-a3ed-bdcea7759da2-maas-api-service-ca\") pod \"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2\" (UID: \"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2\") " Jul 13 17:53:52.195067 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.195043 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a3fdb4-5c84-4e86-a3ed-bdcea7759da2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "99a3fdb4-5c84-4e86-a3ed-bdcea7759da2" (UID: "99a3fdb4-5c84-4e86-a3ed-bdcea7759da2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:53:52.296163 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.296133 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a3fdb4-5c84-4e86-a3ed-bdcea7759da2-maas-api-service-ca\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:53:52.846154 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.846057 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:52.846154 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.846105 2579 patch_prober.go:28] interesting pod/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 13 17:53:52.846445 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.846184 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 13 17:53:52.886773 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.886744 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" Jul 13 17:53:52.887125 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.886774 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-a287d56e-8gptl" event={"ID":"99a3fdb4-5c84-4e86-a3ed-bdcea7759da2","Type":"ContainerDied","Data":"38007340a90b061593f416eb88fdf48837cbd88d0a62357e68fd5c74bc15b098"} Jul 13 17:53:52.887125 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:52.886822 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38007340a90b061593f416eb88fdf48837cbd88d0a62357e68fd5c74bc15b098" Jul 13 17:53:53.846062 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:53.846020 2579 patch_prober.go:28] interesting pod/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 13 17:53:53.846270 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:53.846101 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 13 17:53:54.846154 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:54.846113 2579 patch_prober.go:28] interesting pod/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 13 17:53:54.846666 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:54.846178 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 13 17:53:55.849993 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:55.849965 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:55.850376 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:55.850229 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:53:55.850844 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:53:55.850826 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:54:14.293952 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:14.293915 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8"] Jul 13 17:54:14.294368 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:14.294189 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" containerID="cri-o://1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a" gracePeriod=30 Jul 13 17:54:19.557955 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.557895 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:54:19.736874 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.736848 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-certs\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.736879 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw6nl\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-kube-api-access-cw6nl\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.736898 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-istio-token\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.736922 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-envoy\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.736939 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-socket\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.736960 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-data\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.736986 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-credential-socket\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737347 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.737018 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a80792f0-2878-4c2f-94d7-14edc640333e-istiod-ca-cert\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737347 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.737087 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a80792f0-2878-4c2f-94d7-14edc640333e-istio-podinfo\") pod \"a80792f0-2878-4c2f-94d7-14edc640333e\" (UID: \"a80792f0-2878-4c2f-94d7-14edc640333e\") " Jul 13 17:54:19.737347 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.737159 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:54:19.737347 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.737292 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:54:19.737580 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.737507 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:19.737580 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.737532 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:19.737580 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.737512 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:54:19.737847 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.737824 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a80792f0-2878-4c2f-94d7-14edc640333e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:54:19.738277 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.738249 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:54:19.739268 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.739239 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:54:19.739386 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.739296 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a80792f0-2878-4c2f-94d7-14edc640333e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:54:19.739576 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.739550 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-kube-api-access-cw6nl" (OuterVolumeSpecName: "kube-api-access-cw6nl") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "kube-api-access-cw6nl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:54:19.739672 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.739592 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a80792f0-2878-4c2f-94d7-14edc640333e" (UID: "a80792f0-2878-4c2f-94d7-14edc640333e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:54:19.838232 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.838184 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:19.838232 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.838204 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a80792f0-2878-4c2f-94d7-14edc640333e-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:19.838232 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.838214 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a80792f0-2878-4c2f-94d7-14edc640333e-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:19.838232 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.838224 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw6nl\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-kube-api-access-cw6nl\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:19.838232 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.838233 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a80792f0-2878-4c2f-94d7-14edc640333e-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:19.838463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.838241 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:19.838463 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:19.838248 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a80792f0-2878-4c2f-94d7-14edc640333e-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:54:20.042137 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.042105 2579 generic.go:356] "Generic (PLEG): container finished" podID="a80792f0-2878-4c2f-94d7-14edc640333e" containerID="1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a" exitCode=0 Jul 13 17:54:20.042279 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.042150 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" Jul 13 17:54:20.042279 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.042184 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" event={"ID":"a80792f0-2878-4c2f-94d7-14edc640333e","Type":"ContainerDied","Data":"1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a"} Jul 13 17:54:20.042279 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.042241 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8" event={"ID":"a80792f0-2878-4c2f-94d7-14edc640333e","Type":"ContainerDied","Data":"a21847594a40108e73fd8f0e44360368be576c045776d784d34d7b7514917f70"} Jul 13 17:54:20.042279 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.042267 2579 scope.go:117] "RemoveContainer" containerID="1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a" Jul 13 17:54:20.052148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.052130 2579 scope.go:117] "RemoveContainer" containerID="3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3" Jul 13 17:54:20.060314 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.060296 2579 scope.go:117] "RemoveContainer" containerID="1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a" Jul 13 17:54:20.060583 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:54:20.060564 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a\": container with ID starting with 1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a not found: ID does not exist" containerID="1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a" Jul 13 17:54:20.060657 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.060595 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a"} err="failed to get container status \"1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a\": rpc error: code = NotFound desc = could not find container \"1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a\": container with ID starting with 1f6d1549f3e440722ca60e550c7c406ed0395553accdc9cfe326cd0e9515cc3a not found: ID does not exist" Jul 13 17:54:20.060657 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.060620 2579 scope.go:117] "RemoveContainer" containerID="3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3" Jul 13 17:54:20.060873 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:54:20.060853 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3\": container with ID starting with 3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3 not found: ID does not exist" containerID="3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3" Jul 13 17:54:20.060923 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.060881 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3"} err="failed to get container status \"3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3\": rpc error: code = NotFound desc = could not find container \"3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3\": container with ID starting with 3a1bdf16899d709c4d5ec49bd64c0c0e2675fca987796cb1443625c90cf5c3c3 not found: ID does not exist" Jul 13 17:54:20.064152 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.064132 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8"] Jul 13 17:54:20.069262 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.069241 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-a287d56e-openshift-default-77cfc798c-wjcs8"] Jul 13 17:54:20.669476 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:20.669444 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" path="/var/lib/kubelet/pods/a80792f0-2878-4c2f-94d7-14edc640333e/volumes" Jul 13 17:54:34.547775 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.547744 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs"] Jul 13 17:54:34.548203 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.548189 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" Jul 13 17:54:34.548251 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.548205 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" Jul 13 17:54:34.548251 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.548216 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99a3fdb4-5c84-4e86-a3ed-bdcea7759da2" containerName="revoke-keys" Jul 13 17:54:34.548251 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.548221 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="99a3fdb4-5c84-4e86-a3ed-bdcea7759da2" containerName="revoke-keys" Jul 13 17:54:34.548367 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.548304 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" Jul 13 17:54:34.548367 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.548314 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" Jul 13 17:54:34.548367 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.548339 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="99a3fdb4-5c84-4e86-a3ed-bdcea7759da2" containerName="revoke-keys" Jul 13 17:54:34.552870 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.552853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.555207 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.555189 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-db8658d2-openshift-default-dockercfg-j756z\"" Jul 13 17:54:34.568034 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.568011 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs"] Jul 13 17:54:34.658997 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.658964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-token\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.659178 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.659003 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbc3a849-3092-44e5-bac1-ddcc7752c258-istiod-ca-cert\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.659178 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.659083 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-podinfo\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.659264 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.659169 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h455b\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-kube-api-access-h455b\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.659264 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.659217 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-data\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.659264 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.659239 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-credential-socket\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.659400 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.659279 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-envoy\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.659400 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.659342 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-socket\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.659400 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.659361 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-certs\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.760438 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760410 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-envoy\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.760619 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760458 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-socket\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.760619 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760586 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-certs\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.760705 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-token\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.760787 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760770 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbc3a849-3092-44e5-bac1-ddcc7752c258-istiod-ca-cert\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.760840 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-podinfo\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.760899 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760878 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h455b\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-kube-api-access-h455b\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.760954 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-data\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.761009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760948 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-credential-socket\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.761009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760982 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-certs\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.761009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.760946 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-socket\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.761189 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.761168 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-data\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.761273 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.761253 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-credential-socket\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.761525 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.761504 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbc3a849-3092-44e5-bac1-ddcc7752c258-istiod-ca-cert\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.762738 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.762713 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-envoy\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.763166 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.763147 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-podinfo\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.768435 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.768415 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-token\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.768667 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.768647 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h455b\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-kube-api-access-h455b\") pod \"e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:34.863077 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:34.862981 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:35.013839 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:35.013810 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs"] Jul 13 17:54:35.015091 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:54:35.015060 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbc3a849_3092_44e5_bac1_ddcc7752c258.slice/crio-f0cdcf9f5335a1d34a048fa9935fae050ef1132799dc92462246d5e16af1bf7e WatchSource:0}: Error finding container f0cdcf9f5335a1d34a048fa9935fae050ef1132799dc92462246d5e16af1bf7e: Status 404 returned error can't find the container with id f0cdcf9f5335a1d34a048fa9935fae050ef1132799dc92462246d5e16af1bf7e Jul 13 17:54:35.017341 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:35.017287 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:54:35.017416 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:35.017368 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:54:35.017416 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:35.017396 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:54:35.125399 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:35.125349 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" event={"ID":"fbc3a849-3092-44e5-bac1-ddcc7752c258","Type":"ContainerStarted","Data":"62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12"} Jul 13 17:54:35.125551 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:35.125420 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" event={"ID":"fbc3a849-3092-44e5-bac1-ddcc7752c258","Type":"ContainerStarted","Data":"f0cdcf9f5335a1d34a048fa9935fae050ef1132799dc92462246d5e16af1bf7e"} Jul 13 17:54:35.146895 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:35.146839 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" podStartSLOduration=1.146816706 podStartE2EDuration="1.146816706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:54:35.14588944 +0000 UTC m=+2213.302376316" watchObservedRunningTime="2026-07-13 17:54:35.146816706 +0000 UTC m=+2213.303303582" Jul 13 17:54:35.863638 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:35.863608 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:36.868421 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:36.868385 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:37.137269 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:37.137234 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:37.138247 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:37.138225 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:50.211248 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:50.211181 2579 generic.go:356] "Generic (PLEG): container finished" podID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerID="62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12" exitCode=0 Jul 13 17:54:50.211633 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:50.211254 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" event={"ID":"fbc3a849-3092-44e5-bac1-ddcc7752c258","Type":"ContainerDied","Data":"62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12"} Jul 13 17:54:50.211633 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:50.211602 2579 scope.go:117] "RemoveContainer" containerID="62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12" Jul 13 17:54:50.212539 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:50.212510 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:54:50.212611 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:50.212593 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:54:50.212659 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:50.212645 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:54:51.219057 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:51.219013 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" event={"ID":"fbc3a849-3092-44e5-bac1-ddcc7752c258","Type":"ContainerStarted","Data":"89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18"} Jul 13 17:54:51.863559 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:51.863525 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:51.864631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:51.864611 2579 patch_prober.go:28] interesting pod/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 13 17:54:51.864750 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:51.864653 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 13 17:54:52.137393 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:52.137356 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:52.863774 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:52.863737 2579 patch_prober.go:28] interesting pod/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 13 17:54:52.864244 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:52.863800 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 13 17:54:53.863994 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:53.863956 2579 patch_prober.go:28] interesting pod/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 13 17:54:53.864408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:53.864042 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 13 17:54:54.864096 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:54.864051 2579 patch_prober.go:28] interesting pod/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 13 17:54:54.864481 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:54.864153 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 13 17:54:55.868063 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:55.868033 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:54:55.869159 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:54:55.869136 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:55:24.999663 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:24.999623 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x"] Jul 13 17:55:25.000254 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.000233 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" Jul 13 17:55:25.000352 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.000257 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a80792f0-2878-4c2f-94d7-14edc640333e" containerName="istio-proxy" Jul 13 17:55:25.003493 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.003463 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" Jul 13 17:55:25.005775 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.005758 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-prkcz\"" Jul 13 17:55:25.010770 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.010750 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x"] Jul 13 17:55:25.097634 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.097610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1817f5c-9aab-4a48-99d2-5050c2560fb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-db8658d2-2769x\" (UID: \"b1817f5c-9aab-4a48-99d2-5050c2560fb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" Jul 13 17:55:25.198434 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.198406 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1817f5c-9aab-4a48-99d2-5050c2560fb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-db8658d2-2769x\" (UID: \"b1817f5c-9aab-4a48-99d2-5050c2560fb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" Jul 13 17:55:25.198991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.198970 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1817f5c-9aab-4a48-99d2-5050c2560fb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-db8658d2-2769x\" (UID: \"b1817f5c-9aab-4a48-99d2-5050c2560fb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" Jul 13 17:55:25.315491 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.315435 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" Jul 13 17:55:25.460290 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:25.460265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x"] Jul 13 17:55:25.462391 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:55:25.462359 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1817f5c_9aab_4a48_99d2_5050c2560fb1.slice/crio-8042cc5441b82d7d59ba7520c9852346d650479986f769879f7cd5b0faf80e7c WatchSource:0}: Error finding container 8042cc5441b82d7d59ba7520c9852346d650479986f769879f7cd5b0faf80e7c: Status 404 returned error can't find the container with id 8042cc5441b82d7d59ba7520c9852346d650479986f769879f7cd5b0faf80e7c Jul 13 17:55:26.427804 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:26.427724 2579 generic.go:356] "Generic (PLEG): container finished" podID="b1817f5c-9aab-4a48-99d2-5050c2560fb1" containerID="779e7f15b4ce953ed97fdcf6c33a1b68791325757d0b9c2ed5634962e40b57cf" exitCode=0 Jul 13 17:55:26.427804 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:26.427774 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" event={"ID":"b1817f5c-9aab-4a48-99d2-5050c2560fb1","Type":"ContainerDied","Data":"779e7f15b4ce953ed97fdcf6c33a1b68791325757d0b9c2ed5634962e40b57cf"} Jul 13 17:55:26.428256 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:26.427824 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" event={"ID":"b1817f5c-9aab-4a48-99d2-5050c2560fb1","Type":"ContainerStarted","Data":"8042cc5441b82d7d59ba7520c9852346d650479986f769879f7cd5b0faf80e7c"} Jul 13 17:55:27.568415 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:27.568387 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" Jul 13 17:55:27.617946 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:27.617912 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1817f5c-9aab-4a48-99d2-5050c2560fb1-maas-api-service-ca\") pod \"b1817f5c-9aab-4a48-99d2-5050c2560fb1\" (UID: \"b1817f5c-9aab-4a48-99d2-5050c2560fb1\") " Jul 13 17:55:27.618238 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:27.618214 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1817f5c-9aab-4a48-99d2-5050c2560fb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b1817f5c-9aab-4a48-99d2-5050c2560fb1" (UID: "b1817f5c-9aab-4a48-99d2-5050c2560fb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:55:27.719033 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:27.718962 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1817f5c-9aab-4a48-99d2-5050c2560fb1-maas-api-service-ca\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:55:28.440503 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:28.440470 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" Jul 13 17:55:28.440690 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:28.440516 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-db8658d2-2769x" event={"ID":"b1817f5c-9aab-4a48-99d2-5050c2560fb1","Type":"ContainerDied","Data":"8042cc5441b82d7d59ba7520c9852346d650479986f769879f7cd5b0faf80e7c"} Jul 13 17:55:28.440690 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:55:28.440585 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8042cc5441b82d7d59ba7520c9852346d650479986f769879f7cd5b0faf80e7c" Jul 13 17:56:32.403850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:32.403813 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs"] Jul 13 17:56:32.404419 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:32.404033 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" containerID="cri-o://89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18" gracePeriod=30 Jul 13 17:56:37.668938 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.668879 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:56:37.731909 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.731887 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h455b\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-kube-api-access-h455b\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.731920 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-socket\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.731946 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-token\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.731975 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-data\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.731999 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-credential-socket\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732058 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-certs\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732164 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-envoy\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732177 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:56:37.732253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732193 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-podinfo\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732253 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732248 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbc3a849-3092-44e5-bac1-ddcc7752c258-istiod-ca-cert\") pod \"fbc3a849-3092-44e5-bac1-ddcc7752c258\" (UID: \"fbc3a849-3092-44e5-bac1-ddcc7752c258\") " Jul 13 17:56:37.732695 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732403 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:56:37.732695 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732473 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:56:37.732695 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732645 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.732695 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732670 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.732695 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732685 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.732998 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.732970 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbc3a849-3092-44e5-bac1-ddcc7752c258-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:56:37.733373 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.733347 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-data" (OuterVolumeSpecName: "istio-data") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:56:37.734612 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.734576 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-kube-api-access-h455b" (OuterVolumeSpecName: "kube-api-access-h455b") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "kube-api-access-h455b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:56:37.734790 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.734761 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:56:37.734898 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.734802 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-token" (OuterVolumeSpecName: "istio-token") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:56:37.735020 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.735003 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fbc3a849-3092-44e5-bac1-ddcc7752c258" (UID: "fbc3a849-3092-44e5-bac1-ddcc7752c258"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:56:37.833338 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.833304 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.833478 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.833356 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.833478 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.833369 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbc3a849-3092-44e5-bac1-ddcc7752c258-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.833478 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.833386 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h455b\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-kube-api-access-h455b\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.833478 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.833399 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.833478 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.833412 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbc3a849-3092-44e5-bac1-ddcc7752c258-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:56:37.845662 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.845637 2579 generic.go:356] "Generic (PLEG): container finished" podID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerID="89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18" exitCode=0 Jul 13 17:56:37.845787 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.845685 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" Jul 13 17:56:37.845787 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.845725 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" event={"ID":"fbc3a849-3092-44e5-bac1-ddcc7752c258","Type":"ContainerDied","Data":"89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18"} Jul 13 17:56:37.845910 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.845789 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs" event={"ID":"fbc3a849-3092-44e5-bac1-ddcc7752c258","Type":"ContainerDied","Data":"f0cdcf9f5335a1d34a048fa9935fae050ef1132799dc92462246d5e16af1bf7e"} Jul 13 17:56:37.845910 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.845819 2579 scope.go:117] "RemoveContainer" containerID="89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18" Jul 13 17:56:37.855472 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.855456 2579 scope.go:117] "RemoveContainer" containerID="62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12" Jul 13 17:56:37.863524 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.863509 2579 scope.go:117] "RemoveContainer" containerID="89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18" Jul 13 17:56:37.863790 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:56:37.863764 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18\": container with ID starting with 89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18 not found: ID does not exist" containerID="89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18" Jul 13 17:56:37.863885 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.863799 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18"} err="failed to get container status \"89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18\": rpc error: code = NotFound desc = could not find container \"89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18\": container with ID starting with 89aad48fdc0a6a5cc69e2fc25f47bcc581f571bec7e00aaf186f2bbd96735d18 not found: ID does not exist" Jul 13 17:56:37.863885 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.863820 2579 scope.go:117] "RemoveContainer" containerID="62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12" Jul 13 17:56:37.864020 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:56:37.864006 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12\": container with ID starting with 62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12 not found: ID does not exist" containerID="62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12" Jul 13 17:56:37.864061 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.864023 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12"} err="failed to get container status \"62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12\": rpc error: code = NotFound desc = could not find container \"62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12\": container with ID starting with 62c12436eaa3d1dd5db006c0d98b0862a8786f7f39ea3500db96c569a25c7b12 not found: ID does not exist" Jul 13 17:56:37.868850 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.868821 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs"] Jul 13 17:56:37.873369 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:37.873350 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-db8658d2-openshift-default-f9bcc54dc-fs9hs"] Jul 13 17:56:38.661852 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:56:38.661822 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" path="/var/lib/kubelet/pods/fbc3a849-3092-44e5-bac1-ddcc7752c258/volumes" Jul 13 17:57:29.434115 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.434079 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh"] Jul 13 17:57:29.434586 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.434516 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1817f5c-9aab-4a48-99d2-5050c2560fb1" containerName="revoke-keys" Jul 13 17:57:29.434586 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.434529 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1817f5c-9aab-4a48-99d2-5050c2560fb1" containerName="revoke-keys" Jul 13 17:57:29.434586 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.434541 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" Jul 13 17:57:29.434586 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.434547 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" Jul 13 17:57:29.434740 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.434631 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1817f5c-9aab-4a48-99d2-5050c2560fb1" containerName="revoke-keys" Jul 13 17:57:29.434740 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.434641 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" Jul 13 17:57:29.434740 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.434647 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" Jul 13 17:57:29.437943 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.437921 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.440400 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.440379 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2ffc29-openshift-default-dockercfg-qqzxd\"" Jul 13 17:57:29.463775 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.463744 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh"] Jul 13 17:57:29.562106 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-socket\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.562106 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562106 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istiod-ca-cert\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.562408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562131 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-token\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.562408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562200 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-envoy\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.562408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562273 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-certs\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.562408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-data\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.562408 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562400 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-podinfo\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.562682 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562449 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-credential-socket\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.562682 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.562488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkzjk\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-kube-api-access-xkzjk\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.663581 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663539 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-podinfo\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.663581 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663585 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-credential-socket\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.663810 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663710 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkzjk\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-kube-api-access-xkzjk\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.663810 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663767 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-socket\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.663906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663809 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istiod-ca-cert\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.663906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663846 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-token\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.663906 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663884 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-envoy\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.664059 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663923 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-certs\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.664059 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.663960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-data\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.664059 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.664012 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-credential-socket\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.664227 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.664075 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-socket\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.664285 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.664266 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-certs\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.664387 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.664292 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-data\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.664589 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.664564 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istiod-ca-cert\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.666051 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.666030 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-envoy\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.666140 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.666067 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-podinfo\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.676236 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.676208 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkzjk\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-kube-api-access-xkzjk\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.677889 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.677866 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-token\") pod \"e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.749256 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.749163 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:29.893029 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.892997 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh"] Jul 13 17:57:29.894549 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:57:29.894522 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd535cbd5_6e2f_4aa8_b79b_ce1b6323dcf0.slice/crio-aac8c50c16992fc528345c727ac99ce2b07f47474f799562ed0fe46d7171b661 WatchSource:0}: Error finding container aac8c50c16992fc528345c727ac99ce2b07f47474f799562ed0fe46d7171b661: Status 404 returned error can't find the container with id aac8c50c16992fc528345c727ac99ce2b07f47474f799562ed0fe46d7171b661 Jul 13 17:57:29.896689 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.896656 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:29.896767 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.896721 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:29.896767 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:29.896750 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:30.138519 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:30.138480 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" event={"ID":"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0","Type":"ContainerStarted","Data":"67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25"} Jul 13 17:57:30.138738 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:30.138533 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" event={"ID":"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0","Type":"ContainerStarted","Data":"aac8c50c16992fc528345c727ac99ce2b07f47474f799562ed0fe46d7171b661"} Jul 13 17:57:30.169576 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:30.169519 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" podStartSLOduration=1.16950347 podStartE2EDuration="1.16950347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:57:30.166776777 +0000 UTC m=+2388.323263654" watchObservedRunningTime="2026-07-13 17:57:30.16950347 +0000 UTC m=+2388.325990350" Jul 13 17:57:30.749974 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:30.749941 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:30.754262 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:30.754238 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:31.144280 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:31.144244 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:31.145093 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:31.145075 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:37.292814 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.292779 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff"] Jul 13 17:57:37.293224 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.293210 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" Jul 13 17:57:37.293270 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.293226 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbc3a849-3092-44e5-bac1-ddcc7752c258" containerName="istio-proxy" Jul 13 17:57:37.296896 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.296878 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.300735 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.300716 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-cb4ffa-openshift-default-dockercfg-pd59x\"" Jul 13 17:57:37.325714 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.325689 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff"] Jul 13 17:57:37.439364 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-podinfo\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.439364 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439366 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-data\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.439583 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439389 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05e396da-5911-4321-b4ee-d8dc5151d1f7-istiod-ca-cert\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.439583 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd8l5\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-kube-api-access-nd8l5\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.439583 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439530 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-socket\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.439695 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-credential-socket\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.439695 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439639 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-certs\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.439695 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439682 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-envoy\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.439801 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.439729 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-token\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541015 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.540977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd8l5\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-kube-api-access-nd8l5\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541196 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541024 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-socket\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541196 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-credential-socket\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541196 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-certs\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541196 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-envoy\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541489 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541223 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-token\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541489 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541266 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-podinfo\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541489 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541296 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-data\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541489 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05e396da-5911-4321-b4ee-d8dc5151d1f7-istiod-ca-cert\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541643 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541544 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-credential-socket\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541643 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541565 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-socket\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.541896 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541870 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-data\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.542019 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.541893 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-certs\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.542246 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.542223 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05e396da-5911-4321-b4ee-d8dc5151d1f7-istiod-ca-cert\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.543728 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.543682 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-envoy\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.543794 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.543764 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-podinfo\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.560422 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.560401 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-token\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.564268 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.564245 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd8l5\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-kube-api-access-nd8l5\") pod \"e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.608065 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.608047 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:37.772364 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.772340 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff"] Jul 13 17:57:37.773093 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:57:37.773071 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05e396da_5911_4321_b4ee_d8dc5151d1f7.slice/crio-9b348c28e1649206b44058f167df016b8752918d57b2b66555bb3286aa39c48c WatchSource:0}: Error finding container 9b348c28e1649206b44058f167df016b8752918d57b2b66555bb3286aa39c48c: Status 404 returned error can't find the container with id 9b348c28e1649206b44058f167df016b8752918d57b2b66555bb3286aa39c48c Jul 13 17:57:37.775203 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.775173 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:37.775270 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.775233 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:37.775270 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:37.775257 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:38.196423 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:38.196379 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" event={"ID":"05e396da-5911-4321-b4ee-d8dc5151d1f7","Type":"ContainerStarted","Data":"d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c"} Jul 13 17:57:38.196580 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:38.196467 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" event={"ID":"05e396da-5911-4321-b4ee-d8dc5151d1f7","Type":"ContainerStarted","Data":"9b348c28e1649206b44058f167df016b8752918d57b2b66555bb3286aa39c48c"} Jul 13 17:57:38.232701 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:38.232652 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" podStartSLOduration=1.232637642 podStartE2EDuration="1.232637642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:57:38.229807482 +0000 UTC m=+2396.386294357" watchObservedRunningTime="2026-07-13 17:57:38.232637642 +0000 UTC m=+2396.389124516" Jul 13 17:57:38.609244 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:38.609166 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:38.613993 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:38.613970 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:39.201883 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:39.201854 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:39.202776 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:39.202754 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:42.786628 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:42.786599 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:57:42.788304 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:42.788283 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 17:57:47.186527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:47.186482 2579 patch_prober.go:28] interesting pod/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 17:57:47.187001 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:47.186562 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 17:57:49.269685 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:49.269650 2579 generic.go:356] "Generic (PLEG): container finished" podID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerID="67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25" exitCode=0 Jul 13 17:57:49.270116 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:49.269721 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" event={"ID":"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0","Type":"ContainerDied","Data":"67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25"} Jul 13 17:57:49.270116 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:49.270100 2579 scope.go:117] "RemoveContainer" containerID="67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25" Jul 13 17:57:49.271096 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:49.271064 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:49.271161 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:49.271148 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:49.271194 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:49.271182 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:50.281152 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:50.281112 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" event={"ID":"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0","Type":"ContainerStarted","Data":"db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0"} Jul 13 17:57:50.751588 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:50.750473 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:50.751588 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:50.750647 2579 patch_prober.go:28] interesting pod/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 17:57:50.751588 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:50.750707 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 17:57:51.750259 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:51.750211 2579 patch_prober.go:28] interesting pod/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 17:57:51.750793 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:51.750283 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 17:57:52.750357 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:52.750306 2579 patch_prober.go:28] interesting pod/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 17:57:52.750945 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:52.750911 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 17:57:53.750292 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:53.750247 2579 patch_prober.go:28] interesting pod/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 13 17:57:53.750534 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:53.750338 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 13 17:57:54.753898 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:54.753857 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:54.754354 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:54.754247 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:54.754960 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:54.754939 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:57:55.243527 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:55.243478 2579 patch_prober.go:28] interesting pod/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 13 17:57:55.243729 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:55.243544 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 13 17:57:57.322535 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:57.322457 2579 generic.go:356] "Generic (PLEG): container finished" podID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerID="d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c" exitCode=0 Jul 13 17:57:57.323028 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:57.322533 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" event={"ID":"05e396da-5911-4321-b4ee-d8dc5151d1f7","Type":"ContainerDied","Data":"d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c"} Jul 13 17:57:57.323097 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:57.323080 2579 scope.go:117] "RemoveContainer" containerID="d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c" Jul 13 17:57:57.324395 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:57.324355 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:57.324548 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:57.324453 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:57.324548 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:57.324501 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:57:58.334692 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:58.334653 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" event={"ID":"05e396da-5911-4321-b4ee-d8dc5151d1f7","Type":"ContainerStarted","Data":"d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7"} Jul 13 17:57:58.608371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:58.608260 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:57:58.608540 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:58.608418 2579 patch_prober.go:28] interesting pod/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:57:58.608540 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:58.608463 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:57:59.609461 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:59.609424 2579 patch_prober.go:28] interesting pod/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:57:59.609889 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:57:59.609487 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:58:00.609036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:58:00.608996 2579 patch_prober.go:28] interesting pod/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:58:00.609216 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:58:00.609067 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:58:01.609433 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:58:01.609397 2579 patch_prober.go:28] interesting pod/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 13 17:58:01.609820 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:58:01.609453 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 13 17:58:02.612895 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:58:02.612864 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:58:02.613261 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:58:02.613225 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:58:02.614026 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:58:02.614010 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:59:15.425819 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:15.425786 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh"] Jul 13 17:59:15.426351 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:15.426036 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" containerID="cri-o://db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0" gracePeriod=30 Jul 13 17:59:20.678438 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.678414 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:59:20.750794 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.750727 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-envoy\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.750794 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.750780 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-socket\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.750943 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.750823 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-data\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.750943 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.750858 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-token\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.750943 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.750883 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-credential-socket\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.750943 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.750932 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkzjk\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-kube-api-access-xkzjk\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.751212 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.750971 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-podinfo\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.751212 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751005 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istiod-ca-cert\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.751212 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751055 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:20.751212 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751066 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-certs\") pod \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\" (UID: \"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0\") " Jul 13 17:59:20.751472 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751219 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:20.751472 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751281 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:20.751955 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751620 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:59:20.751955 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751641 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:20.751955 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751662 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:20.751955 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751696 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:20.751955 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.751865 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-data" (OuterVolumeSpecName: "istio-data") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:20.753056 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.753017 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:20.753215 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.753194 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-token" (OuterVolumeSpecName: "istio-token") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:59:20.753313 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.753296 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:59:20.753469 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.753452 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-kube-api-access-xkzjk" (OuterVolumeSpecName: "kube-api-access-xkzjk") pod "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" (UID: "d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0"). InnerVolumeSpecName "kube-api-access-xkzjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:59:20.813865 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.813834 2579 generic.go:356] "Generic (PLEG): container finished" podID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerID="db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0" exitCode=0 Jul 13 17:59:20.813991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.813887 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" Jul 13 17:59:20.813991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.813908 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" event={"ID":"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0","Type":"ContainerDied","Data":"db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0"} Jul 13 17:59:20.813991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.813962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh" event={"ID":"d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0","Type":"ContainerDied","Data":"aac8c50c16992fc528345c727ac99ce2b07f47474f799562ed0fe46d7171b661"} Jul 13 17:59:20.813991 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.813984 2579 scope.go:117] "RemoveContainer" containerID="db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0" Jul 13 17:59:20.823276 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.823257 2579 scope.go:117] "RemoveContainer" containerID="67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25" Jul 13 17:59:20.832098 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.832082 2579 scope.go:117] "RemoveContainer" containerID="db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0" Jul 13 17:59:20.832363 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:59:20.832338 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0\": container with ID starting with db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0 not found: ID does not exist" containerID="db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0" Jul 13 17:59:20.832410 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.832375 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0"} err="failed to get container status \"db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0\": rpc error: code = NotFound desc = could not find container \"db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0\": container with ID starting with db19f7bac98b3f003c509375ea352e4be077b361ec5a8c75e862b8b46d4a5ef0 not found: ID does not exist" Jul 13 17:59:20.832410 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.832399 2579 scope.go:117] "RemoveContainer" containerID="67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25" Jul 13 17:59:20.832639 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:59:20.832621 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25\": container with ID starting with 67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25 not found: ID does not exist" containerID="67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25" Jul 13 17:59:20.832726 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.832645 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25"} err="failed to get container status \"67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25\": rpc error: code = NotFound desc = could not find container \"67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25\": container with ID starting with 67acf572df5bc58a44ae59248ab1d608401ab94e6688218cc0f7ed983ae75b25 not found: ID does not exist" Jul 13 17:59:20.836166 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.836147 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh"] Jul 13 17:59:20.840368 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.840301 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2ffc29-openshift-default-7587c848f7-dkrlh"] Jul 13 17:59:20.852371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.852350 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:20.852371 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.852368 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:20.852494 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.852376 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:20.852494 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.852385 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkzjk\" (UniqueName: \"kubernetes.io/projected/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-kube-api-access-xkzjk\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:20.852494 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.852393 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:20.852494 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:20.852401 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:22.661827 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:22.661792 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" path="/var/lib/kubelet/pods/d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0/volumes" Jul 13 17:59:41.553910 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:41.553872 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff"] Jul 13 17:59:41.554519 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:41.554098 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" containerID="cri-o://d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7" gracePeriod=30 Jul 13 17:59:42.457413 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.457377 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk"] Jul 13 17:59:42.457814 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.457800 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" Jul 13 17:59:42.457858 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.457815 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" Jul 13 17:59:42.457900 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.457891 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" Jul 13 17:59:42.457941 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.457903 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" Jul 13 17:59:42.462554 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.462533 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.465062 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.465042 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f033d9-openshift-default-dockercfg-k889w\"" Jul 13 17:59:42.476072 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.476048 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk"] Jul 13 17:59:42.637158 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637127 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-token\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.637631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637165 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-data\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.637631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637245 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-envoy\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.637631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637288 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-certs\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.637631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637311 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-socket\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.637631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637439 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptt9q\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-kube-api-access-ptt9q\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.637631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-credential-socket\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.637631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-podinfo\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.637631 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.637553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istiod-ca-cert\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.738886 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.738809 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-envoy\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.738886 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.738849 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-certs\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.738886 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.738868 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-socket\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739142 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.738909 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptt9q\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-kube-api-access-ptt9q\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739142 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.738934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-credential-socket\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739142 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.738950 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-podinfo\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739142 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.738977 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istiod-ca-cert\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739142 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.739062 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-token\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739142 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.739092 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-data\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739499 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.739313 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-certs\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739499 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.739370 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-credential-socket\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739499 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.739405 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-socket\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739499 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.739490 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-data\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.739886 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.739858 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istiod-ca-cert\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.741148 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.741126 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-envoy\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.741805 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.741780 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-podinfo\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.747105 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.747077 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-token\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.747199 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.747180 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptt9q\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-kube-api-access-ptt9q\") pod \"e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.772147 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.772124 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:42.913839 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.913811 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk"] Jul 13 17:59:42.914933 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:59:42.914899 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58053e9_f0f6_4de6_8fce_9e3320e36a9c.slice/crio-43fe4d9fb7c923ebf23630e0447b42cfd1b69cf716fb1bc19aeec5f2000bf10f WatchSource:0}: Error finding container 43fe4d9fb7c923ebf23630e0447b42cfd1b69cf716fb1bc19aeec5f2000bf10f: Status 404 returned error can't find the container with id 43fe4d9fb7c923ebf23630e0447b42cfd1b69cf716fb1bc19aeec5f2000bf10f Jul 13 17:59:42.916985 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.916948 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:59:42.917065 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.917015 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:59:42.917065 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.917043 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:59:42.944384 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:42.944359 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" event={"ID":"d58053e9-f0f6-4de6-8fce-9e3320e36a9c","Type":"ContainerStarted","Data":"43fe4d9fb7c923ebf23630e0447b42cfd1b69cf716fb1bc19aeec5f2000bf10f"} Jul 13 17:59:43.952266 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:43.952230 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" event={"ID":"d58053e9-f0f6-4de6-8fce-9e3320e36a9c","Type":"ContainerStarted","Data":"85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e"} Jul 13 17:59:43.972455 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:43.972409 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" podStartSLOduration=1.9723929999999998 podStartE2EDuration="1.972393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:59:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:59:43.971377415 +0000 UTC m=+2522.127864291" watchObservedRunningTime="2026-07-13 17:59:43.972393 +0000 UTC m=+2522.128879874" Jul 13 17:59:44.773222 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:44.773189 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:44.777719 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:44.777692 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:44.957555 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:44.957526 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:44.958468 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:44.958452 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 17:59:46.812405 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.812378 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:59:46.874469 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874445 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-data\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874621 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874489 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05e396da-5911-4321-b4ee-d8dc5151d1f7-istiod-ca-cert\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874621 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874527 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-token\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874621 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874593 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-socket\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874741 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874641 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd8l5\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-kube-api-access-nd8l5\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874741 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874680 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-podinfo\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874835 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874764 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-credential-socket\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874835 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874790 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-envoy\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874945 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874863 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-certs\") pod \"05e396da-5911-4321-b4ee-d8dc5151d1f7\" (UID: \"05e396da-5911-4321-b4ee-d8dc5151d1f7\") " Jul 13 17:59:46.874945 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874890 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:46.875050 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.874952 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05e396da-5911-4321-b4ee-d8dc5151d1f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 17:59:46.875050 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.875027 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:46.875158 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.875122 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:46.875300 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.875281 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.875397 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.875305 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.875397 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.875344 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05e396da-5911-4321-b4ee-d8dc5151d1f7-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.875397 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.875358 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.875545 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.875418 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:46.876868 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.876840 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:59:46.876986 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.876897 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 17:59:46.877264 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.877246 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-kube-api-access-nd8l5" (OuterVolumeSpecName: "kube-api-access-nd8l5") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "kube-api-access-nd8l5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 17:59:46.877405 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.877385 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05e396da-5911-4321-b4ee-d8dc5151d1f7" (UID: "05e396da-5911-4321-b4ee-d8dc5151d1f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 17:59:46.970228 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.970199 2579 generic.go:356] "Generic (PLEG): container finished" podID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerID="d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7" exitCode=0 Jul 13 17:59:46.970404 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.970246 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" Jul 13 17:59:46.970404 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.970282 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" event={"ID":"05e396da-5911-4321-b4ee-d8dc5151d1f7","Type":"ContainerDied","Data":"d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7"} Jul 13 17:59:46.970404 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.970372 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff" event={"ID":"05e396da-5911-4321-b4ee-d8dc5151d1f7","Type":"ContainerDied","Data":"9b348c28e1649206b44058f167df016b8752918d57b2b66555bb3286aa39c48c"} Jul 13 17:59:46.970574 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.970407 2579 scope.go:117] "RemoveContainer" containerID="d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7" Jul 13 17:59:46.976046 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.976021 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.976181 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.976049 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.976181 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.976064 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.976181 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.976077 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.976181 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.976089 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd8l5\" (UniqueName: \"kubernetes.io/projected/05e396da-5911-4321-b4ee-d8dc5151d1f7-kube-api-access-nd8l5\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 17:59:46.980625 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.980608 2579 scope.go:117] "RemoveContainer" containerID="d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c" Jul 13 17:59:46.989126 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.989108 2579 scope.go:117] "RemoveContainer" containerID="d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7" Jul 13 17:59:46.989406 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:59:46.989387 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7\": container with ID starting with d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7 not found: ID does not exist" containerID="d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7" Jul 13 17:59:46.989460 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.989416 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7"} err="failed to get container status \"d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7\": rpc error: code = NotFound desc = could not find container \"d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7\": container with ID starting with d4e885d42615d8093ee8ec522b5344839a3a31a38530bd14a04dd2bcab8faad7 not found: ID does not exist" Jul 13 17:59:46.989460 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.989436 2579 scope.go:117] "RemoveContainer" containerID="d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c" Jul 13 17:59:46.989661 ip-10-0-143-131 kubenswrapper[2579]: E0713 17:59:46.989643 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c\": container with ID starting with d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c not found: ID does not exist" containerID="d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c" Jul 13 17:59:46.989703 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.989667 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c"} err="failed to get container status \"d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c\": rpc error: code = NotFound desc = could not find container \"d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c\": container with ID starting with d25f91fd2575c7f78d740c72233fd72f516ad42b536369d8733e6dec63001c6c not found: ID does not exist" Jul 13 17:59:46.996210 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:46.996186 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff"] Jul 13 17:59:47.000685 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:47.000666 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-cb4ffa-openshift-default-5d5fd57d4c-69kff"] Jul 13 17:59:48.662100 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:48.662069 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" path="/var/lib/kubelet/pods/05e396da-5911-4321-b4ee-d8dc5151d1f7/volumes" Jul 13 17:59:51.186766 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.186730 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl"] Jul 13 17:59:51.187240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.187122 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" Jul 13 17:59:51.187240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.187134 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d535cbd5-6e2f-4aa8-b79b-ce1b6323dcf0" containerName="istio-proxy" Jul 13 17:59:51.187240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.187151 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" Jul 13 17:59:51.187240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.187157 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" Jul 13 17:59:51.187240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.187166 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" Jul 13 17:59:51.187240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.187171 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" Jul 13 17:59:51.187240 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.187237 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" Jul 13 17:59:51.190368 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.190346 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.193245 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.193226 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d6c068-openshift-default-dockercfg-6rmnw\"" Jul 13 17:59:51.202999 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.202977 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl"] Jul 13 17:59:51.212748 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.212729 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-envoy\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.212865 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.212764 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-token\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.212865 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.212784 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnjht\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-kube-api-access-wnjht\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.212865 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.212810 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-data\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.212865 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.212851 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117c88a7-f461-43a6-9b9d-014664eb40fd-istiod-ca-cert\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.213036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.212890 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-socket\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.213036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.212924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-credential-socket\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.213036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.212964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-podinfo\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.213036 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.213001 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-certs\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.313954 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.313919 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-certs\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.313970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-envoy\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.313996 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-token\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314013 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnjht\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-kube-api-access-wnjht\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314031 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-data\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314049 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117c88a7-f461-43a6-9b9d-014664eb40fd-istiod-ca-cert\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314094 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-socket\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314155 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-credential-socket\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314563 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-podinfo\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314563 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-certs\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314563 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314480 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-data\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314738 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314635 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-credential-socket\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314738 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-socket\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.314845 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.314701 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117c88a7-f461-43a6-9b9d-014664eb40fd-istiod-ca-cert\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.316652 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.316619 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-envoy\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.316776 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.316678 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-podinfo\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.330482 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.322728 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-token\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.330482 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.323116 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnjht\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-kube-api-access-wnjht\") pod \"e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.501650 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.501571 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:51.653301 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.653268 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl"] Jul 13 17:59:51.654042 ip-10-0-143-131 kubenswrapper[2579]: W0713 17:59:51.654019 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117c88a7_f461_43a6_9b9d_014664eb40fd.slice/crio-7fb865a99477b64756f3af84c96968d6c08077092e09714355e082fdabc2cfa5 WatchSource:0}: Error finding container 7fb865a99477b64756f3af84c96968d6c08077092e09714355e082fdabc2cfa5: Status 404 returned error can't find the container with id 7fb865a99477b64756f3af84c96968d6c08077092e09714355e082fdabc2cfa5 Jul 13 17:59:51.656616 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.656581 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:59:51.656696 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.656646 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:59:51.656696 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:51.656674 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 17:59:52.001569 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:52.001528 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" event={"ID":"117c88a7-f461-43a6-9b9d-014664eb40fd","Type":"ContainerStarted","Data":"556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314"} Jul 13 17:59:52.001772 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:52.001590 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" event={"ID":"117c88a7-f461-43a6-9b9d-014664eb40fd","Type":"ContainerStarted","Data":"7fb865a99477b64756f3af84c96968d6c08077092e09714355e082fdabc2cfa5"} Jul 13 17:59:52.024193 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:52.024136 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" podStartSLOduration=1.02411651 podStartE2EDuration="1.02411651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:59:52.020996226 +0000 UTC m=+2530.177483102" watchObservedRunningTime="2026-07-13 17:59:52.02411651 +0000 UTC m=+2530.180603387" Jul 13 17:59:52.502589 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:52.502536 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:52.507773 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:52.507746 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:53.007009 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:53.006974 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:53.008018 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:53.007993 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 17:59:59.959172 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:59.959117 2579 patch_prober.go:28] interesting pod/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 17:59:59.959633 ip-10-0-143-131 kubenswrapper[2579]: I0713 17:59:59.959197 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 18:00:00.057403 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:00.057366 2579 generic.go:356] "Generic (PLEG): container finished" podID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerID="85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e" exitCode=0 Jul 13 18:00:00.057576 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:00.057455 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" event={"ID":"d58053e9-f0f6-4de6-8fce-9e3320e36a9c","Type":"ContainerDied","Data":"85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e"} Jul 13 18:00:00.057938 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:00.057920 2579 scope.go:117] "RemoveContainer" containerID="85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e" Jul 13 18:00:00.059155 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:00.059118 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:00:00.059265 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:00.059204 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:00:00.059265 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:00.059237 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:00:01.068152 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:01.068112 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" event={"ID":"d58053e9-f0f6-4de6-8fce-9e3320e36a9c","Type":"ContainerStarted","Data":"8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701"} Jul 13 18:00:01.772864 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:01.772828 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 18:00:01.773215 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:01.773193 2579 patch_prober.go:28] interesting pod/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 18:00:01.773393 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:01.773367 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 18:00:02.773399 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:02.773360 2579 patch_prober.go:28] interesting pod/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 18:00:02.773971 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:02.773940 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 18:00:03.772814 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:03.772770 2579 patch_prober.go:28] interesting pod/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 18:00:03.773027 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:03.772840 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 18:00:04.772505 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:04.772467 2579 patch_prober.go:28] interesting pod/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 13 18:00:04.772982 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:04.772521 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 13 18:00:05.777227 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:05.777194 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 18:00:05.777685 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:05.777588 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 18:00:05.778215 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:05.778196 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 18:00:07.108941 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:07.108912 2579 generic.go:356] "Generic (PLEG): container finished" podID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerID="556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314" exitCode=0 Jul 13 18:00:07.109288 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:07.108986 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" event={"ID":"117c88a7-f461-43a6-9b9d-014664eb40fd","Type":"ContainerDied","Data":"556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314"} Jul 13 18:00:07.109545 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:07.109531 2579 scope.go:117] "RemoveContainer" containerID="556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314" Jul 13 18:00:07.110648 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:07.110618 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:00:07.110730 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:07.110700 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:00:07.110769 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:07.110749 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:00:08.006911 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:08.006876 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 18:00:08.118220 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:08.118183 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" event={"ID":"117c88a7-f461-43a6-9b9d-014664eb40fd","Type":"ContainerStarted","Data":"cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92"} Jul 13 18:00:08.502771 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:08.502737 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 18:00:08.503621 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:08.503597 2579 patch_prober.go:28] interesting pod/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 13 18:00:08.503677 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:08.503650 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 13 18:00:09.502718 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:09.502683 2579 patch_prober.go:28] interesting pod/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 13 18:00:09.503103 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:09.502765 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 13 18:00:10.502213 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:10.502119 2579 patch_prober.go:28] interesting pod/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 13 18:00:10.502213 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:10.502186 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 13 18:00:11.502700 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:11.502664 2579 patch_prober.go:28] interesting pod/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 13 18:00:11.503078 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:11.502720 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 13 18:00:12.506667 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:12.506632 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 18:00:12.507056 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:12.506908 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 18:00:12.507687 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:12.507670 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 18:00:32.459144 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:32.459107 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk"] Jul 13 18:00:32.459696 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:32.459397 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" containerID="cri-o://8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701" gracePeriod=30 Jul 13 18:00:35.779497 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:35.779463 2579 patch_prober.go:28] interesting pod/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 18:00:35.779991 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:35.779528 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 18:00:37.710501 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.710476 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 18:00:37.792362 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792280 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-envoy\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792362 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792342 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-token\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792545 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792364 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istiod-ca-cert\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792545 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792422 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-certs\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792545 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792441 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-credential-socket\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792545 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792468 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-data\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792545 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792501 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptt9q\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-kube-api-access-ptt9q\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792545 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792541 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-socket\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792853 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792592 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-podinfo\") pod \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\" (UID: \"d58053e9-f0f6-4de6-8fce-9e3320e36a9c\") " Jul 13 18:00:37.792853 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792724 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:00:37.792853 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792742 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:00:37.792853 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792839 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:00:37.793051 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.792965 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:00:37.793117 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.793097 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:37.793177 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.793121 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:37.793177 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.793133 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:37.793177 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.793145 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:37.793482 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.793459 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:00:37.794762 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.794740 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-kube-api-access-ptt9q" (OuterVolumeSpecName: "kube-api-access-ptt9q") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "kube-api-access-ptt9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:00:37.794762 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.794749 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 18:00:37.794978 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.794958 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:00:37.795022 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.794959 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d58053e9-f0f6-4de6-8fce-9e3320e36a9c" (UID: "d58053e9-f0f6-4de6-8fce-9e3320e36a9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:00:37.894168 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.894142 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:37.894168 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.894167 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptt9q\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-kube-api-access-ptt9q\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:37.894308 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.894176 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:37.894308 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.894185 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:37.894308 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:37.894192 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d58053e9-f0f6-4de6-8fce-9e3320e36a9c-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:00:38.322007 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.321975 2579 generic.go:356] "Generic (PLEG): container finished" podID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerID="8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701" exitCode=0 Jul 13 18:00:38.322168 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.322021 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" Jul 13 18:00:38.322168 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.322061 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" event={"ID":"d58053e9-f0f6-4de6-8fce-9e3320e36a9c","Type":"ContainerDied","Data":"8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701"} Jul 13 18:00:38.322168 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.322122 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk" event={"ID":"d58053e9-f0f6-4de6-8fce-9e3320e36a9c","Type":"ContainerDied","Data":"43fe4d9fb7c923ebf23630e0447b42cfd1b69cf716fb1bc19aeec5f2000bf10f"} Jul 13 18:00:38.322168 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.322152 2579 scope.go:117] "RemoveContainer" containerID="8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701" Jul 13 18:00:38.332295 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.332278 2579 scope.go:117] "RemoveContainer" containerID="85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e" Jul 13 18:00:38.340708 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.340692 2579 scope.go:117] "RemoveContainer" containerID="8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701" Jul 13 18:00:38.340973 ip-10-0-143-131 kubenswrapper[2579]: E0713 18:00:38.340954 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701\": container with ID starting with 8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701 not found: ID does not exist" containerID="8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701" Jul 13 18:00:38.341036 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.340980 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701"} err="failed to get container status \"8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701\": rpc error: code = NotFound desc = could not find container \"8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701\": container with ID starting with 8b07b570c4b5c242c6fd8148965f824dad8752acc89acebb6cd26554a7aba701 not found: ID does not exist" Jul 13 18:00:38.341036 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.340998 2579 scope.go:117] "RemoveContainer" containerID="85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e" Jul 13 18:00:38.341233 ip-10-0-143-131 kubenswrapper[2579]: E0713 18:00:38.341216 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e\": container with ID starting with 85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e not found: ID does not exist" containerID="85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e" Jul 13 18:00:38.341276 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.341239 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e"} err="failed to get container status \"85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e\": rpc error: code = NotFound desc = could not find container \"85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e\": container with ID starting with 85e65d9f6aa14f480dcaa68097fb8c99cf2cc00b4354dd5488d63ac0e422022e not found: ID does not exist" Jul 13 18:00:38.345936 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.345914 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk"] Jul 13 18:00:38.349798 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.349778 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f033d9-openshift-default-78494949d5-vqqvk"] Jul 13 18:00:38.662485 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:38.662456 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" path="/var/lib/kubelet/pods/d58053e9-f0f6-4de6-8fce-9e3320e36a9c/volumes" Jul 13 18:00:58.670806 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:58.670773 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl"] Jul 13 18:00:58.671400 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:58.671024 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" containerID="cri-o://cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92" gracePeriod=30 Jul 13 18:00:59.611682 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.611652 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf"] Jul 13 18:00:59.612077 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.612063 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" Jul 13 18:00:59.612133 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.612079 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" Jul 13 18:00:59.612133 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.612098 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" Jul 13 18:00:59.612133 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.612104 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" Jul 13 18:00:59.612244 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.612166 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="05e396da-5911-4321-b4ee-d8dc5151d1f7" containerName="istio-proxy" Jul 13 18:00:59.612244 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.612173 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" Jul 13 18:00:59.612244 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.612180 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58053e9-f0f6-4de6-8fce-9e3320e36a9c" containerName="istio-proxy" Jul 13 18:00:59.615382 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.615365 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.617928 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.617911 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f54db3-openshift-default-dockercfg-7hp9j\"" Jul 13 18:00:59.630970 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.630944 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf"] Jul 13 18:00:59.680832 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.680803 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-token\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.681185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.680866 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-credential-socket\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.681185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.680924 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istiod-ca-cert\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.681185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.680960 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-podinfo\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.681185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.681015 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-data\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.681185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.681060 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-envoy\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.681185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.681085 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-socket\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.681185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.681122 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7ksf\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-kube-api-access-x7ksf\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.681185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.681139 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-certs\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782119 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782086 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7ksf\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-kube-api-access-x7ksf\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782119 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782118 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-certs\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782449 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-token\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782449 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-credential-socket\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782449 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istiod-ca-cert\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782449 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782230 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-podinfo\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782449 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782263 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-data\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782449 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-envoy\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782449 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782379 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-socket\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782810 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782556 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-certs\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782810 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782763 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-credential-socket\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782810 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782781 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-data\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782925 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782881 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-socket\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.782960 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.782942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istiod-ca-cert\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.784578 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.784551 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-envoy\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.784925 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.784901 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-podinfo\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.790443 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.790420 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-token\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.790800 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.790776 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7ksf\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-kube-api-access-x7ksf\") pod \"e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:00:59.926899 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:00:59.926866 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:00.078187 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:00.078156 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf"] Jul 13 18:01:00.079848 ip-10-0-143-131 kubenswrapper[2579]: W0713 18:01:00.079822 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf81d0337_c784_43b0_85bb_41e4d95b5f5d.slice/crio-1608d3cc0b1dc1318caaa340e7e42c187da70b34f5377ede75e4881d25f1867a WatchSource:0}: Error finding container 1608d3cc0b1dc1318caaa340e7e42c187da70b34f5377ede75e4881d25f1867a: Status 404 returned error can't find the container with id 1608d3cc0b1dc1318caaa340e7e42c187da70b34f5377ede75e4881d25f1867a Jul 13 18:01:00.082546 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:00.082509 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:00.082631 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:00.082584 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:00.082631 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:00.082621 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:00.460776 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:00.460743 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" event={"ID":"f81d0337-c784-43b0-85bb-41e4d95b5f5d","Type":"ContainerStarted","Data":"2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913"} Jul 13 18:01:00.460942 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:00.460799 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" event={"ID":"f81d0337-c784-43b0-85bb-41e4d95b5f5d","Type":"ContainerStarted","Data":"1608d3cc0b1dc1318caaa340e7e42c187da70b34f5377ede75e4881d25f1867a"} Jul 13 18:01:00.489070 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:00.489015 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" podStartSLOduration=1.4889981190000001 podStartE2EDuration="1.488998119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:01:00.486515037 +0000 UTC m=+2598.643001912" watchObservedRunningTime="2026-07-13 18:01:00.488998119 +0000 UTC m=+2598.645484993" Jul 13 18:01:00.927005 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:00.926974 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:01.932838 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:01.932809 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:02.472761 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:02.472730 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:02.473829 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:02.473808 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:03.933759 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:03.933696 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 18:01:04.021973 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.021942 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-podinfo\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022141 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022027 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-certs\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022141 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022090 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-envoy\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022265 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022149 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-token\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022265 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022179 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnjht\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-kube-api-access-wnjht\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022265 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022246 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-credential-socket\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022461 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022282 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117c88a7-f461-43a6-9b9d-014664eb40fd-istiod-ca-cert\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022461 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022310 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-data\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022461 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022314 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:01:04.022461 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022384 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-socket\") pod \"117c88a7-f461-43a6-9b9d-014664eb40fd\" (UID: \"117c88a7-f461-43a6-9b9d-014664eb40fd\") " Jul 13 18:01:04.022674 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022578 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:01:04.022733 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022682 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.022791 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022766 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/117c88a7-f461-43a6-9b9d-014664eb40fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:01:04.022896 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.022874 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:01:04.023412 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.023369 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:01:04.024740 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.024526 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 18:01:04.024740 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.024571 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:01:04.025533 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.025475 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-kube-api-access-wnjht" (OuterVolumeSpecName: "kube-api-access-wnjht") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "kube-api-access-wnjht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:01:04.025533 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.025513 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "117c88a7-f461-43a6-9b9d-014664eb40fd" (UID: "117c88a7-f461-43a6-9b9d-014664eb40fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:01:04.123746 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.123682 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.123746 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.123715 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/117c88a7-f461-43a6-9b9d-014664eb40fd-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.123746 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.123730 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.123746 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.123743 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.124013 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.123757 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.124013 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.123770 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.124013 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.123783 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.124013 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.123796 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnjht\" (UniqueName: \"kubernetes.io/projected/117c88a7-f461-43a6-9b9d-014664eb40fd-kube-api-access-wnjht\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:01:04.485311 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.485272 2579 generic.go:356] "Generic (PLEG): container finished" podID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerID="cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92" exitCode=0 Jul 13 18:01:04.485515 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.485339 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" Jul 13 18:01:04.485515 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.485364 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" event={"ID":"117c88a7-f461-43a6-9b9d-014664eb40fd","Type":"ContainerDied","Data":"cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92"} Jul 13 18:01:04.485515 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.485444 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl" event={"ID":"117c88a7-f461-43a6-9b9d-014664eb40fd","Type":"ContainerDied","Data":"7fb865a99477b64756f3af84c96968d6c08077092e09714355e082fdabc2cfa5"} Jul 13 18:01:04.485515 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.485467 2579 scope.go:117] "RemoveContainer" containerID="cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92" Jul 13 18:01:04.497052 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.497032 2579 scope.go:117] "RemoveContainer" containerID="556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314" Jul 13 18:01:04.506372 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.506303 2579 scope.go:117] "RemoveContainer" containerID="cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92" Jul 13 18:01:04.506812 ip-10-0-143-131 kubenswrapper[2579]: E0713 18:01:04.506693 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92\": container with ID starting with cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92 not found: ID does not exist" containerID="cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92" Jul 13 18:01:04.506812 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.506747 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92"} err="failed to get container status \"cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92\": rpc error: code = NotFound desc = could not find container \"cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92\": container with ID starting with cb5d0db111ea293fe236d30ddb85d75d9ea9a35100fe53feb9b675eb0e144a92 not found: ID does not exist" Jul 13 18:01:04.506812 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.506775 2579 scope.go:117] "RemoveContainer" containerID="556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314" Jul 13 18:01:04.507126 ip-10-0-143-131 kubenswrapper[2579]: E0713 18:01:04.507104 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314\": container with ID starting with 556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314 not found: ID does not exist" containerID="556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314" Jul 13 18:01:04.507176 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.507134 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314"} err="failed to get container status \"556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314\": rpc error: code = NotFound desc = could not find container \"556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314\": container with ID starting with 556441cdceac0056d30d091f09134c1e24e9bb51be23a9253397ddf22ed4e314 not found: ID does not exist" Jul 13 18:01:04.510058 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.509927 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl"] Jul 13 18:01:04.513040 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.513021 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d6c068-openshift-default-5c5c8fbf59-mxhfl"] Jul 13 18:01:04.661462 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:04.661430 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" path="/var/lib/kubelet/pods/117c88a7-f461-43a6-9b9d-014664eb40fd/volumes" Jul 13 18:01:07.610044 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.610010 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x"] Jul 13 18:01:07.610571 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.610553 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" Jul 13 18:01:07.610571 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.610572 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" Jul 13 18:01:07.610662 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.610647 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" Jul 13 18:01:07.610662 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.610657 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" Jul 13 18:01:07.615439 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.615418 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.617651 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.617628 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b57346-openshift-default-dockercfg-899fp\"" Jul 13 18:01:07.650425 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.650395 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x"] Jul 13 18:01:07.653837 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.653811 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-socket\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.653925 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.653867 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-token\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.653925 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.653900 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-certs\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.654016 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.653946 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a840558e-dca1-411d-ac55-0adc8ea9093f-istiod-ca-cert\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.654070 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.654043 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-envoy\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.654106 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.654091 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-credential-socket\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.654141 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.654125 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-data\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.654180 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.654146 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-podinfo\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.654180 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.654174 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69f9m\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-kube-api-access-69f9m\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755378 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-envoy\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755546 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-credential-socket\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755546 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755436 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-data\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755546 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755462 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-podinfo\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755546 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69f9m\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-kube-api-access-69f9m\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755772 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755626 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-socket\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755772 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755685 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-token\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755772 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755743 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-certs\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755953 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755792 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a840558e-dca1-411d-ac55-0adc8ea9093f-istiod-ca-cert\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.755953 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.755803 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-credential-socket\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.756204 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.756176 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-certs\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.756428 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.756408 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-data\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.756520 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.756417 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-socket\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.756520 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.756506 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a840558e-dca1-411d-ac55-0adc8ea9093f-istiod-ca-cert\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.757759 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.757739 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-envoy\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.757975 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.757957 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-podinfo\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.769293 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.769267 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69f9m\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-kube-api-access-69f9m\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.771533 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.771510 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-token\") pod \"e2e-shared-b-b57346-openshift-default-6467877555-vr25x\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:07.927518 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:07.927476 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:08.072542 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:08.072512 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x"] Jul 13 18:01:08.073619 ip-10-0-143-131 kubenswrapper[2579]: W0713 18:01:08.073588 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda840558e_dca1_411d_ac55_0adc8ea9093f.slice/crio-2306f7cad2a080b99e34d81797737ba73148c71a6801daadf0498e41e55b8b53 WatchSource:0}: Error finding container 2306f7cad2a080b99e34d81797737ba73148c71a6801daadf0498e41e55b8b53: Status 404 returned error can't find the container with id 2306f7cad2a080b99e34d81797737ba73148c71a6801daadf0498e41e55b8b53 Jul 13 18:01:08.075803 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:08.075771 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:08.075854 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:08.075841 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:08.075896 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:08.075872 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:08.514594 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:08.514548 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" event={"ID":"a840558e-dca1-411d-ac55-0adc8ea9093f","Type":"ContainerStarted","Data":"06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9"} Jul 13 18:01:08.514808 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:08.514628 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" event={"ID":"a840558e-dca1-411d-ac55-0adc8ea9093f","Type":"ContainerStarted","Data":"2306f7cad2a080b99e34d81797737ba73148c71a6801daadf0498e41e55b8b53"} Jul 13 18:01:08.544888 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:08.544828 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" podStartSLOduration=1.5448123310000001 podStartE2EDuration="1.544812331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:01:08.533491888 +0000 UTC m=+2606.689978763" watchObservedRunningTime="2026-07-13 18:01:08.544812331 +0000 UTC m=+2606.701299203" Jul 13 18:01:08.927985 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:08.927953 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:09.944034 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:09.943986 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:10.528678 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:10.528650 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:10.535837 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:10.535582 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:17.473663 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:17.473629 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 18:01:17.474084 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:17.473687 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 18:01:17.588483 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:17.588447 2579 generic.go:356] "Generic (PLEG): container finished" podID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerID="2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913" exitCode=0 Jul 13 18:01:17.588661 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:17.588482 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" event={"ID":"f81d0337-c784-43b0-85bb-41e4d95b5f5d","Type":"ContainerDied","Data":"2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913"} Jul 13 18:01:17.588926 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:17.588912 2579 scope.go:117] "RemoveContainer" containerID="2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913" Jul 13 18:01:17.590075 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:17.590043 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:17.590145 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:17.590124 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:17.590201 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:17.590163 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:18.596582 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:18.596542 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" event={"ID":"f81d0337-c784-43b0-85bb-41e4d95b5f5d","Type":"ContainerStarted","Data":"e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f"} Jul 13 18:01:18.927580 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:18.927544 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:18.927783 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:18.927670 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 18:01:18.927783 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:18.927716 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 18:01:19.927446 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:19.927398 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 18:01:19.928125 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:19.928087 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 18:01:20.927836 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:20.927800 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 18:01:20.928456 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:20.928424 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 18:01:21.928416 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:21.927935 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 18:01:21.928416 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:21.928010 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 18:01:22.928108 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:22.928070 2579 patch_prober.go:28] interesting pod/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 13 18:01:22.928424 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:22.928393 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 13 18:01:23.931554 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:23.931523 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:23.932081 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:23.932064 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:23.932837 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:23.932812 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:01:25.530024 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:25.529987 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b57346-openshift-default-6467877555-vr25x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 18:01:25.530467 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:25.530043 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 18:01:25.653434 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:25.653399 2579 generic.go:356] "Generic (PLEG): container finished" podID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerID="06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9" exitCode=0 Jul 13 18:01:25.653593 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:25.653479 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" event={"ID":"a840558e-dca1-411d-ac55-0adc8ea9093f","Type":"ContainerDied","Data":"06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9"} Jul 13 18:01:25.654028 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:25.654005 2579 scope.go:117] "RemoveContainer" containerID="06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9" Jul 13 18:01:25.655226 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:25.655197 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:25.655306 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:25.655260 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:25.655306 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:25.655291 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 18:01:26.663898 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:26.663865 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" event={"ID":"a840558e-dca1-411d-ac55-0adc8ea9093f","Type":"ContainerStarted","Data":"97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409"} Jul 13 18:01:26.928629 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:26.928525 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:26.928815 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:26.928666 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b57346-openshift-default-6467877555-vr25x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 18:01:26.928815 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:26.928716 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 18:01:27.928506 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:27.928464 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b57346-openshift-default-6467877555-vr25x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 18:01:27.929001 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:27.928550 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 18:01:28.927896 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:28.927854 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b57346-openshift-default-6467877555-vr25x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 18:01:28.928102 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:28.927933 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 18:01:29.928110 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:29.928070 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b57346-openshift-default-6467877555-vr25x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 13 18:01:29.928743 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:29.928709 2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 13 18:01:30.932944 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:30.932909 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:30.933428 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:30.933301 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:01:30.933994 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:01:30.933976 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:02:42.824982 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:02:42.824952 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 18:02:42.827050 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:02:42.827026 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 18:07:42.863098 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:07:42.863065 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 18:07:42.866042 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:07:42.866022 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-snh5v_ac6adcdd-a342-4652-8762-af45ca4d1114/ovn-acl-logging/0.log" Jul 13 18:08:02.363486 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:02.363450 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf"] Jul 13 18:08:02.363926 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:02.363681 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" containerID="cri-o://e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f" gracePeriod=30 Jul 13 18:08:07.617682 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.617649 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:08:07.699856 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.699799 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istiod-ca-cert\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.699856 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.699831 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-envoy\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.699856 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.699853 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-socket\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.700075 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.699880 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-podinfo\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.700075 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.699952 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-credential-socket\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.700075 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.699978 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7ksf\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-kube-api-access-x7ksf\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.700075 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700006 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-certs\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.700075 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700044 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-data\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.700346 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700083 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-token\") pod \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\" (UID: \"f81d0337-c784-43b0-85bb-41e4d95b5f5d\") " Jul 13 18:08:07.700346 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700157 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:07.700346 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700199 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:07.700346 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700212 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:08:07.700575 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700400 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:07.700575 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700525 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.700575 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700545 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.700575 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700557 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.700575 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.700569 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.701086 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.701059 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-data" (OuterVolumeSpecName: "istio-data") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:07.702255 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.702227 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-token" (OuterVolumeSpecName: "istio-token") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:08:07.702391 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.702255 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 18:08:07.702447 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.702386 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:07.702546 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.702521 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-kube-api-access-x7ksf" (OuterVolumeSpecName: "kube-api-access-x7ksf") pod "f81d0337-c784-43b0-85bb-41e4d95b5f5d" (UID: "f81d0337-c784-43b0-85bb-41e4d95b5f5d"). InnerVolumeSpecName "kube-api-access-x7ksf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:08:07.801427 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.801404 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7ksf\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-kube-api-access-x7ksf\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.801427 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.801425 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.801551 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.801434 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.801551 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.801442 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.801551 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.801449 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f81d0337-c784-43b0-85bb-41e4d95b5f5d-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:07.974237 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.974176 2579 generic.go:356] "Generic (PLEG): container finished" podID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerID="e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f" exitCode=0 Jul 13 18:08:07.974237 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.974224 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" Jul 13 18:08:07.974385 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.974246 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" event={"ID":"f81d0337-c784-43b0-85bb-41e4d95b5f5d","Type":"ContainerDied","Data":"e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f"} Jul 13 18:08:07.974385 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.974286 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf" event={"ID":"f81d0337-c784-43b0-85bb-41e4d95b5f5d","Type":"ContainerDied","Data":"1608d3cc0b1dc1318caaa340e7e42c187da70b34f5377ede75e4881d25f1867a"} Jul 13 18:08:07.974385 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.974306 2579 scope.go:117] "RemoveContainer" containerID="e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f" Jul 13 18:08:07.983707 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.983691 2579 scope.go:117] "RemoveContainer" containerID="2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913" Jul 13 18:08:07.991671 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.991646 2579 scope.go:117] "RemoveContainer" containerID="e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f" Jul 13 18:08:07.991928 ip-10-0-143-131 kubenswrapper[2579]: E0713 18:08:07.991904 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f\": container with ID starting with e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f not found: ID does not exist" containerID="e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f" Jul 13 18:08:07.992006 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.991939 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f"} err="failed to get container status \"e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f\": rpc error: code = NotFound desc = could not find container \"e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f\": container with ID starting with e507913305d89363baa532cdf36a54ac07844d5942b4b2123e363fd72ce0dd5f not found: ID does not exist" Jul 13 18:08:07.992006 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.991961 2579 scope.go:117] "RemoveContainer" containerID="2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913" Jul 13 18:08:07.992200 ip-10-0-143-131 kubenswrapper[2579]: E0713 18:08:07.992183 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913\": container with ID starting with 2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913 not found: ID does not exist" containerID="2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913" Jul 13 18:08:07.992243 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.992205 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913"} err="failed to get container status \"2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913\": rpc error: code = NotFound desc = could not find container \"2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913\": container with ID starting with 2a59f63b288d39c48c0a121805286175128062700447177346a4a50041d79913 not found: ID does not exist" Jul 13 18:08:07.996901 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:07.996881 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf"] Jul 13 18:08:08.001315 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:08.001294 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f54db3-openshift-default-6bc6d7599-4kjdf"] Jul 13 18:08:08.661756 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:08.661726 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" path="/var/lib/kubelet/pods/f81d0337-c784-43b0-85bb-41e4d95b5f5d/volumes" Jul 13 18:08:28.587363 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:28.587332 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x"] Jul 13 18:08:28.587887 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:28.587567 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" containerID="cri-o://97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409" gracePeriod=30 Jul 13 18:08:30.936127 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:30.936088 2579 patch_prober.go:28] interesting pod/e2e-shared-b-b57346-openshift-default-6467877555-vr25x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 18:08:30.936541 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:30.936164 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 18:08:33.849453 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:33.849392 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:08:34.035470 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035438 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-credential-socket\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035640 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035508 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-socket\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035640 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035537 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a840558e-dca1-411d-ac55-0adc8ea9093f-istiod-ca-cert\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035640 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035560 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-envoy\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035640 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035593 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-data\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035640 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035618 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-certs\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035922 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035664 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69f9m\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-kube-api-access-69f9m\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035922 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035717 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-podinfo\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035922 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035734 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:34.035922 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035752 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-token\") pod \"a840558e-dca1-411d-ac55-0adc8ea9093f\" (UID: \"a840558e-dca1-411d-ac55-0adc8ea9093f\") " Jul 13 18:08:34.035922 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035843 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:34.035922 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.035901 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:34.036223 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.036202 2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.036283 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.036228 2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-workload-certs\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.036283 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.036243 2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-credential-socket\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.036283 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.036236 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a840558e-dca1-411d-ac55-0adc8ea9093f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 18:08:34.036558 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.036520 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:34.037990 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.037955 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-kube-api-access-69f9m" (OuterVolumeSpecName: "kube-api-access-69f9m") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "kube-api-access-69f9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:08:34.038094 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.037986 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 18:08:34.038094 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.038025 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 18:08:34.038094 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.038063 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a840558e-dca1-411d-ac55-0adc8ea9093f" (UID: "a840558e-dca1-411d-ac55-0adc8ea9093f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 13 18:08:34.122136 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.122044 2579 generic.go:356] "Generic (PLEG): container finished" podID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerID="97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409" exitCode=0 Jul 13 18:08:34.122136 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.122102 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" Jul 13 18:08:34.122287 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.122099 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" event={"ID":"a840558e-dca1-411d-ac55-0adc8ea9093f","Type":"ContainerDied","Data":"97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409"} Jul 13 18:08:34.122287 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.122258 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x" event={"ID":"a840558e-dca1-411d-ac55-0adc8ea9093f","Type":"ContainerDied","Data":"2306f7cad2a080b99e34d81797737ba73148c71a6801daadf0498e41e55b8b53"} Jul 13 18:08:34.122381 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.122288 2579 scope.go:117] "RemoveContainer" containerID="97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409" Jul 13 18:08:34.132668 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.132653 2579 scope.go:117] "RemoveContainer" containerID="06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9" Jul 13 18:08:34.137545 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.137526 2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a840558e-dca1-411d-ac55-0adc8ea9093f-istiod-ca-cert\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.137626 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.137549 2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-envoy\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.137626 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.137561 2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-data\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.137626 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.137570 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69f9m\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-kube-api-access-69f9m\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.137626 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.137578 2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-podinfo\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.137626 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.137587 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a840558e-dca1-411d-ac55-0adc8ea9093f-istio-token\") on node \"ip-10-0-143-131.ec2.internal\" DevicePath \"\"" Jul 13 18:08:34.143520 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.143500 2579 scope.go:117] "RemoveContainer" containerID="97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409" Jul 13 18:08:34.143979 ip-10-0-143-131 kubenswrapper[2579]: E0713 18:08:34.143850 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409\": container with ID starting with 97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409 not found: ID does not exist" containerID="97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409" Jul 13 18:08:34.143979 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.143887 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409"} err="failed to get container status \"97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409\": rpc error: code = NotFound desc = could not find container \"97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409\": container with ID starting with 97cc3042d9489027d076e365e362777e4d49bf7c232ab8405d44cb23b5b80409 not found: ID does not exist" Jul 13 18:08:34.143979 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.143912 2579 scope.go:117] "RemoveContainer" containerID="06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9" Jul 13 18:08:34.144249 ip-10-0-143-131 kubenswrapper[2579]: E0713 18:08:34.144226 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9\": container with ID starting with 06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9 not found: ID does not exist" containerID="06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9" Jul 13 18:08:34.144314 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.144259 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9"} err="failed to get container status \"06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9\": rpc error: code = NotFound desc = could not find container \"06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9\": container with ID starting with 06e7b81ed4ff96830cf2be258b9960d9ec81993541e1589a56b1012e7ce7a7d9 not found: ID does not exist" Jul 13 18:08:34.146186 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.146165 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x"] Jul 13 18:08:34.151265 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.151244 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b57346-openshift-default-6467877555-vr25x"] Jul 13 18:08:34.552080 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.552048 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rh45f_2bca07f1-0dc8-452c-ab58-b49b7479fc07/manager/0.log" Jul 13 18:08:34.661683 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.661647 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" path="/var/lib/kubelet/pods/a840558e-dca1-411d-ac55-0adc8ea9093f/volumes" Jul 13 18:08:34.674433 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.674393 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-x5ptt_436dc897-f260-4a0c-a8b3-fe6aff676d24/manager/0.log" Jul 13 18:08:34.963121 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:34.963089 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lk78q_22e7f71a-9d76-4daf-9356-6822b3d39f6b/server/0.log" Jul 13 18:08:35.528232 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:35.528200 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx_62232eb5-da3c-4c6c-be28-50363cd0cf32/copy-manifests/0.log" Jul 13 18:08:35.535465 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:35.535439 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx_62232eb5-da3c-4c6c-be28-50363cd0cf32/manager/0.log" Jul 13 18:08:35.909263 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:35.909231 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-868b79db4-wrtlq_1747b87b-bebe-44fd-9d19-a257ef47b2dd/maas-api/0.log" Jul 13 18:08:36.751617 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:36.751593 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-7c24d8cc-xb4pf_f68be85d-b295-4600-b10f-7d5a64edb27f/revoke-keys/0.log" Jul 13 18:08:37.101266 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:37.101188 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-35a3687f-v56q7_d0dc835f-82b3-4131-a5a1-93fc14a17d52/revoke-keys/0.log" Jul 13 18:08:37.224711 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:37.224681 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-a287d56e-8gptl_99a3fdb4-5c84-4e86-a3ed-bdcea7759da2/revoke-keys/0.log" Jul 13 18:08:37.348634 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:37.348595 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-db8658d2-2769x_b1817f5c-9aab-4a48-99d2-5050c2560fb1/revoke-keys/0.log" Jul 13 18:08:38.528031 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.527994 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s_51b2cced-8bce-490f-953d-5070575075c7/util/0.log" Jul 13 18:08:38.534686 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.534664 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s_51b2cced-8bce-490f-953d-5070575075c7/pull/0.log" Jul 13 18:08:38.541142 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.541125 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s_51b2cced-8bce-490f-953d-5070575075c7/extract/0.log" Jul 13 18:08:38.666171 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.666149 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p_855e3210-cac3-4dcb-a191-2f839797472e/util/0.log" Jul 13 18:08:38.673686 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.673660 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p_855e3210-cac3-4dcb-a191-2f839797472e/pull/0.log" Jul 13 18:08:38.682503 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.682481 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p_855e3210-cac3-4dcb-a191-2f839797472e/extract/0.log" Jul 13 18:08:38.804475 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.804413 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x_d074c6fb-0915-410c-81ca-39303c43cecf/util/0.log" Jul 13 18:08:38.817063 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.817041 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x_d074c6fb-0915-410c-81ca-39303c43cecf/pull/0.log" Jul 13 18:08:38.826188 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.826171 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x_d074c6fb-0915-410c-81ca-39303c43cecf/extract/0.log" Jul 13 18:08:38.939587 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.939558 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm_6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f/util/0.log" Jul 13 18:08:38.948805 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.948780 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm_6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f/pull/0.log" Jul 13 18:08:38.955704 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:38.955686 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm_6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f/extract/0.log" Jul 13 18:08:39.672972 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:39.672929 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q2djd_e334d773-74b6-41d4-b1bc-d0c0e4fb8f81/manager/0.log" Jul 13 18:08:40.295225 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:40.295193 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45l7t7_2af08a2d-6765-4d43-a546-9c11fbd32823/istio-proxy/0.log" Jul 13 18:08:40.422087 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:40.422059 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xr4bj_9551e061-4d3f-4ede-9d70-50d88a56dc94/discovery/0.log" Jul 13 18:08:40.543963 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:40.543933 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c65495b5-64fdt_1e516dff-e9c7-4106-ae22-260904fd849d/kube-auth-proxy/0.log" Jul 13 18:08:40.793086 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:40.793055 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rk8z7_f348f376-d3b1-4b27-b342-236e5b92e9a3/istio-proxy/1.log" Jul 13 18:08:41.174199 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:41.174172 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56768b9b-6x2s5_6714944a-56b5-4b88-b10e-22fd57415282/router/0.log" Jul 13 18:08:41.678162 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:41.678124 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-jsxtm_3f6400a4-644a-4d31-8570-b2e58eb70925/main/0.log" Jul 13 18:08:41.920777 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:41.920743 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbkrbng_e1adfb80-fc2c-4c56-924d-a2824f8a013a/main/0.log" Jul 13 18:08:49.944191 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:49.944161 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-twrr7_50f4cbd8-b73c-4a9f-8e83-4e192a9fc5d9/global-pull-secret-syncer/0.log" Jul 13 18:08:50.059668 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:50.059644 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z96wx_13d2222c-9fec-4ae0-849a-49f4bf8b21e3/konnectivity-agent/0.log" Jul 13 18:08:50.124053 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:50.124023 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-131.ec2.internal_e6d457d90ba56949705c8ce1904d5667/haproxy/0.log" Jul 13 18:08:54.345377 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.345274 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s_51b2cced-8bce-490f-953d-5070575075c7/extract/0.log" Jul 13 18:08:54.378717 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.378697 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s_51b2cced-8bce-490f-953d-5070575075c7/util/0.log" Jul 13 18:08:54.402015 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.401978 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759jdw5s_51b2cced-8bce-490f-953d-5070575075c7/pull/0.log" Jul 13 18:08:54.435941 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.435914 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p_855e3210-cac3-4dcb-a191-2f839797472e/extract/0.log" Jul 13 18:08:54.466029 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.466004 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p_855e3210-cac3-4dcb-a191-2f839797472e/util/0.log" Jul 13 18:08:54.489069 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.489046 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e056p2p_855e3210-cac3-4dcb-a191-2f839797472e/pull/0.log" Jul 13 18:08:54.520529 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.520508 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x_d074c6fb-0915-410c-81ca-39303c43cecf/extract/0.log" Jul 13 18:08:54.542433 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.542412 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x_d074c6fb-0915-410c-81ca-39303c43cecf/util/0.log" Jul 13 18:08:54.566719 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.566698 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73rm82x_d074c6fb-0915-410c-81ca-39303c43cecf/pull/0.log" Jul 13 18:08:54.593925 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.593898 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm_6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f/extract/0.log" Jul 13 18:08:54.614807 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.614746 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm_6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f/util/0.log" Jul 13 18:08:54.637190 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:54.637173 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1w7dgm_6d5a9bb6-c8b2-4576-9e30-cec9695a7e6f/pull/0.log" Jul 13 18:08:55.325332 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:55.325301 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-q2djd_e334d773-74b6-41d4-b1bc-d0c0e4fb8f81/manager/0.log" Jul 13 18:08:57.225569 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.225465 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-6nfw2_cc37ddc9-06d8-4427-b437-aff8b7169805/cluster-monitoring-operator/0.log" Jul 13 18:08:57.256092 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.256062 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-x7k6v_5e2a4f47-4904-4460-9672-59da0994d1ed/kube-state-metrics/0.log" Jul 13 18:08:57.278024 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.278001 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-x7k6v_5e2a4f47-4904-4460-9672-59da0994d1ed/kube-rbac-proxy-main/0.log" Jul 13 18:08:57.327123 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.327101 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-x7k6v_5e2a4f47-4904-4460-9672-59da0994d1ed/kube-rbac-proxy-self/0.log" Jul 13 18:08:57.412957 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.412912 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-fff5f5d57-fl8pw_dba3fa76-12cb-4cef-81d8-1598a4474c1b/metrics-server/0.log" Jul 13 18:08:57.439881 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.439856 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-svsrf_50d8ede4-66fb-44c4-9324-740e23226fec/monitoring-plugin/0.log" Jul 13 18:08:57.465929 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.465902 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2v56r_ae89211d-8087-4afe-9718-6dc001e42016/node-exporter/0.log" Jul 13 18:08:57.484569 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.484509 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2v56r_ae89211d-8087-4afe-9718-6dc001e42016/kube-rbac-proxy/0.log" Jul 13 18:08:57.508452 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.508435 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2v56r_ae89211d-8087-4afe-9718-6dc001e42016/init-textfile/0.log" Jul 13 18:08:57.726305 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.726272 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4q4kk_73fc4818-1724-4006-b1b4-650ef9b5f41d/kube-rbac-proxy-main/0.log" Jul 13 18:08:57.751387 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.751295 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4q4kk_73fc4818-1724-4006-b1b4-650ef9b5f41d/kube-rbac-proxy-self/0.log" Jul 13 18:08:57.774791 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.774762 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-4q4kk_73fc4818-1724-4006-b1b4-650ef9b5f41d/openshift-state-metrics/0.log" Jul 13 18:08:57.985486 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:57.985452 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jqrt7_f397236d-344d-4817-bfc7-f0c3c9f6b0d0/prometheus-operator/0.log" Jul 13 18:08:58.004096 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:58.004007 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-jqrt7_f397236d-344d-4817-bfc7-f0c3c9f6b0d0/kube-rbac-proxy/0.log" Jul 13 18:08:59.785488 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.785457 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj"] Jul 13 18:08:59.785920 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.785903 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" Jul 13 18:08:59.785920 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.785921 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" Jul 13 18:08:59.786000 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.785936 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" Jul 13 18:08:59.786000 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.785941 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" Jul 13 18:08:59.786000 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.785971 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" Jul 13 18:08:59.786000 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.785980 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="117c88a7-f461-43a6-9b9d-014664eb40fd" containerName="istio-proxy" Jul 13 18:08:59.786000 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.785993 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" Jul 13 18:08:59.786000 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.786001 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" Jul 13 18:08:59.786197 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.786008 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" Jul 13 18:08:59.786197 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.786014 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" Jul 13 18:08:59.786197 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.786098 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f81d0337-c784-43b0-85bb-41e4d95b5f5d" containerName="istio-proxy" Jul 13 18:08:59.786197 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.786107 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" Jul 13 18:08:59.786197 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.786115 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a840558e-dca1-411d-ac55-0adc8ea9093f" containerName="istio-proxy" Jul 13 18:08:59.789291 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.789270 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.791679 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.791654 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dg5t9\"/\"kube-root-ca.crt\"" Jul 13 18:08:59.791781 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.791665 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dg5t9\"/\"openshift-service-ca.crt\"" Jul 13 18:08:59.792444 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.792427 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dg5t9\"/\"default-dockercfg-7czpt\"" Jul 13 18:08:59.800931 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.800913 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj"] Jul 13 18:08:59.837993 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.837964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-proc\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.838136 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.838019 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-podres\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.838136 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.838073 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlgs2\" (UniqueName: \"kubernetes.io/projected/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-kube-api-access-nlgs2\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.838136 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.838106 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-sys\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.838275 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.838143 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-lib-modules\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939116 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939088 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-sys\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939261 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939149 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-lib-modules\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939261 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939191 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-proc\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939261 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939212 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-sys\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939261 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-podres\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939439 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939296 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlgs2\" (UniqueName: \"kubernetes.io/projected/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-kube-api-access-nlgs2\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939439 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939364 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-lib-modules\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939439 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939380 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-proc\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.939439 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.939411 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-podres\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:08:59.947527 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:08:59.947507 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlgs2\" (UniqueName: \"kubernetes.io/projected/1e8a58da-fd4b-40a7-8a6a-e85be35659fd-kube-api-access-nlgs2\") pod \"perf-node-gather-daemonset-kpdtj\" (UID: \"1e8a58da-fd4b-40a7-8a6a-e85be35659fd\") " pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:09:00.100173 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:00.100138 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:09:00.235025 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:00.234995 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj"] Jul 13 18:09:00.236093 ip-10-0-143-131 kubenswrapper[2579]: W0713 18:09:00.236067 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1e8a58da_fd4b_40a7_8a6a_e85be35659fd.slice/crio-b12914bea55b7fd0ae656272551e854e4d02095bb452db961f7e52b96623aade WatchSource:0}: Error finding container b12914bea55b7fd0ae656272551e854e4d02095bb452db961f7e52b96623aade: Status 404 returned error can't find the container with id b12914bea55b7fd0ae656272551e854e4d02095bb452db961f7e52b96623aade Jul 13 18:09:00.271208 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:00.271181 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" event={"ID":"1e8a58da-fd4b-40a7-8a6a-e85be35659fd","Type":"ContainerStarted","Data":"b12914bea55b7fd0ae656272551e854e4d02095bb452db961f7e52b96623aade"} Jul 13 18:09:00.680035 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:00.680003 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-tw5tt_56e2f450-991e-43cc-bfdf-3078b3768f44/download-server/0.log" Jul 13 18:09:01.264020 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:01.263984 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-sfp7j_6054271a-bcd5-495c-b204-70feb3e81753/volume-data-source-validator/0.log" Jul 13 18:09:01.278655 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:01.278621 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" event={"ID":"1e8a58da-fd4b-40a7-8a6a-e85be35659fd","Type":"ContainerStarted","Data":"54236f38d48c22b532dbdf7785aacf4a7fbd009800e44bf12c7617b56a2f0560"} Jul 13 18:09:01.278824 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:01.278745 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:09:01.296951 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:01.296906 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" podStartSLOduration=2.296892498 podStartE2EDuration="2.296892498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:09:01.29473741 +0000 UTC m=+3079.451224288" watchObservedRunningTime="2026-07-13 18:09:01.296892498 +0000 UTC m=+3079.453379373" Jul 13 18:09:02.342994 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:02.342962 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cdlrj_9baab0f5-91c6-4a40-89b4-241bd5585973/dns/0.log" Jul 13 18:09:02.363944 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:02.363920 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cdlrj_9baab0f5-91c6-4a40-89b4-241bd5585973/kube-rbac-proxy/0.log" Jul 13 18:09:02.437149 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:02.437125 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kl9vm_cf7543c5-daf6-4ad6-9d05-d00958baac07/dns-node-resolver/0.log" Jul 13 18:09:03.028572 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:03.028541 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bqkvc_35d03ed5-a235-40aa-815f-b8f286ba77c3/node-ca/0.log" Jul 13 18:09:04.020849 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:04.020817 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45l7t7_2af08a2d-6765-4d43-a546-9c11fbd32823/istio-proxy/0.log" Jul 13 18:09:04.143695 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:04.143662 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xr4bj_9551e061-4d3f-4ede-9d70-50d88a56dc94/discovery/0.log" Jul 13 18:09:04.164579 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:04.164548 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c65495b5-64fdt_1e516dff-e9c7-4106-ae22-260904fd849d/kube-auth-proxy/0.log" Jul 13 18:09:04.225334 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:04.225297 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rk8z7_f348f376-d3b1-4b27-b342-236e5b92e9a3/istio-proxy/0.log" Jul 13 18:09:04.259922 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:04.259893 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rk8z7_f348f376-d3b1-4b27-b342-236e5b92e9a3/istio-proxy/1.log" Jul 13 18:09:04.340778 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:04.340697 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56768b9b-6x2s5_6714944a-56b5-4b88-b10e-22fd57415282/router/0.log" Jul 13 18:09:05.060359 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:05.060316 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7qt69_535200a1-6053-4333-8463-07781e5f0486/serve-healthcheck-canary/0.log" Jul 13 18:09:05.619036 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:05.619001 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hql5g_7a9856f9-2976-466d-b07f-e43582a7d59a/insights-operator/0.log" Jul 13 18:09:05.619638 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:05.619619 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hql5g_7a9856f9-2976-466d-b07f-e43582a7d59a/insights-operator/1.log" Jul 13 18:09:05.641969 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:05.641946 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-289jk_4caee1b8-5d63-4814-b089-967a32b2e038/kube-rbac-proxy/0.log" Jul 13 18:09:05.663762 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:05.663740 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-289jk_4caee1b8-5d63-4814-b089-967a32b2e038/exporter/0.log" Jul 13 18:09:05.686424 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:05.686405 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-289jk_4caee1b8-5d63-4814-b089-967a32b2e038/extractor/0.log" Jul 13 18:09:07.295750 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:07.295717 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dg5t9/perf-node-gather-daemonset-kpdtj" Jul 13 18:09:08.248456 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:08.248422 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-rh45f_2bca07f1-0dc8-452c-ab58-b49b7479fc07/manager/0.log" Jul 13 18:09:08.403472 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:08.403445 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-x5ptt_436dc897-f260-4a0c-a8b3-fe6aff676d24/manager/0.log" Jul 13 18:09:08.532358 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:08.532247 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-lk78q_22e7f71a-9d76-4daf-9356-6822b3d39f6b/server/0.log" Jul 13 18:09:08.847177 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:08.847144 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx_62232eb5-da3c-4c6c-be28-50363cd0cf32/manager/0.log" Jul 13 18:09:08.867263 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:08.867241 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc4cfb8b6-tl8tx_62232eb5-da3c-4c6c-be28-50363cd0cf32/copy-manifests/0.log" Jul 13 18:09:09.880731 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:09.880698 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45l7t7_2af08a2d-6765-4d43-a546-9c11fbd32823/istio-proxy/0.log" Jul 13 18:09:09.995260 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:09.995231 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xr4bj_9551e061-4d3f-4ede-9d70-50d88a56dc94/discovery/0.log" Jul 13 18:09:10.006306 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:10.006283 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c65495b5-64fdt_1e516dff-e9c7-4106-ae22-260904fd849d/kube-auth-proxy/0.log" Jul 13 18:09:10.047185 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:10.047168 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rk8z7_f348f376-d3b1-4b27-b342-236e5b92e9a3/istio-proxy/0.log" Jul 13 18:09:10.076092 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:10.076072 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rk8z7_f348f376-d3b1-4b27-b342-236e5b92e9a3/istio-proxy/1.log" Jul 13 18:09:10.137941 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:10.137922 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56768b9b-6x2s5_6714944a-56b5-4b88-b10e-22fd57415282/router/0.log" Jul 13 18:09:11.338900 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:11.338867 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45l7t7_2af08a2d-6765-4d43-a546-9c11fbd32823/istio-proxy/0.log" Jul 13 18:09:11.453957 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:11.453911 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-xr4bj_9551e061-4d3f-4ede-9d70-50d88a56dc94/discovery/0.log" Jul 13 18:09:11.465906 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:11.465883 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65c65495b5-64fdt_1e516dff-e9c7-4106-ae22-260904fd849d/kube-auth-proxy/0.log" Jul 13 18:09:11.505955 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:11.505927 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rk8z7_f348f376-d3b1-4b27-b342-236e5b92e9a3/istio-proxy/0.log" Jul 13 18:09:11.540816 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:11.540790 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rk8z7_f348f376-d3b1-4b27-b342-236e5b92e9a3/istio-proxy/1.log" Jul 13 18:09:11.603497 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:11.603473 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56768b9b-6x2s5_6714944a-56b5-4b88-b10e-22fd57415282/router/0.log" Jul 13 18:09:15.240061 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:15.240032 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-qf4wb_3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e/migrator/0.log" Jul 13 18:09:15.262307 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:15.262276 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-qf4wb_3d0dfe0e-61f4-4f6f-8e1d-c8274642cd2e/graceful-termination/0.log" Jul 13 18:09:16.735077 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:16.735050 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cwbb8_bd72547b-0a43-40f8-8ec6-ea60f1634e41/kube-multus-additional-cni-plugins/0.log" Jul 13 18:09:16.755785 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:16.755756 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cwbb8_bd72547b-0a43-40f8-8ec6-ea60f1634e41/egress-router-binary-copy/0.log" Jul 13 18:09:16.775881 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:16.775858 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cwbb8_bd72547b-0a43-40f8-8ec6-ea60f1634e41/cni-plugins/0.log" Jul 13 18:09:16.796404 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:16.796388 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cwbb8_bd72547b-0a43-40f8-8ec6-ea60f1634e41/bond-cni-plugin/0.log" Jul 13 18:09:16.817402 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:16.817382 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cwbb8_bd72547b-0a43-40f8-8ec6-ea60f1634e41/routeoverride-cni/0.log" Jul 13 18:09:16.839588 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:16.839570 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cwbb8_bd72547b-0a43-40f8-8ec6-ea60f1634e41/whereabouts-cni-bincopy/0.log" Jul 13 18:09:16.863563 ip-10-0-143-131 kubenswrapper[2579]: I0713 18:09:16.863548 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cwbb8_bd72547b-0a43-40f8-8ec6-ea60f1634e41/whereabouts-cni/0.log"